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EDITOR »S  INTRODUCTION 


The  first  time  he  meets  matrices,,  the  student  of  numerical  analysis  is 
not  usually  reader  for  a systematic  development  of  elementary  divisor  theory 
and  still  less  for  a treatment  over  a general  field,  But  he  does  need  to 
learn  the  main  facts  about  the  algebra  of  finite  matrices  over  the  real  or 
complex  field,  and  tie  should  know  something  about  the  proofs  „ These  matters 
are  supplied  very  well  by  Faddeeva  in  the  chapter  translated  here,  and  the 
editor  therefore  believes  that  the  chapter  will  be  very  popular 0 

Analogous  statements  could  be  made  about  the  analytical  aspects  of 
finite  matrix  theory s norms,  limit,  etc. 

In  the  two  later  chapters  of  her  book,  the  author  describes  various 
known  numerical  processes  for  inverting  matrices  and  for  obtaining  their 
latent  roots  and  vectors,  and  gives  mary  numerical  examples e These  chapters 
probably  also  merit  translation,  and  we  hope  later  to  be  able  to  provide  one. 
The  first  chapter  was  translated  specifically  to  provide  an  introduc- 
tory text  for  the  editor 8 s 19,^2  summer  course  on  numerical  matrix  methods 
at  the  University  of  California,  Los  Angeles , With  this  in  view,  the 
translator  has  done  more  than  create  an  interesting  and  faithful  transla- 
tions he  has  improved  the  presentation  in  several  respects.  He  has  cor- 
rected misprints,  and  has  helped  the  student  by  occasionally  inserting 
phrases  and  by  adding  two  paragraphs  of  recapitulation  (ppc  UU  and  y5)0 
He  has  changed  a few  notations  to  suit  American  taste.  Finally,  he  has 
replaced  the  Russian  bibliography  by  an  English  and  American  one,  and  has 
considerably  supplemented  the  author ®s  three  or  four  references  to  the 
bibliography  e 

Only  the  more  significant  of  the  translator  * s additions  have  been 
credited  to  him  in  the  text. 
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AUTHOR’S  PREFACE 


The  numerical  solution  of  the  problems  of  mathematical  physics  is 
most  frequently  connected  with  the  numerical  solution  of  basic  problems  of 
linear  algebra  — that  of  solving  a system  of  linear  equations , and  that  of 
the  computation  of  the  latent  roots  of  a matrix.  The  present  book  is  an 
endeavor  at  systematising  the  most  important  numerical  methods  of  linear 
algebra  — the  classical  ones  and  those  elaborated  quite  recently  as  well® 

The  author  does  not  pretend  to  an  exhaustive  completeness,  having  in- 
cluded an  exposition  of  those  methods  only  that  have  already  been  tested 
in  practice.  In  the  exposition  the  author  has  not  strived  for  an  irreproach- 
able rigor,  and  has  not  analysed  all  conceivable  cases  and  sub-cases  arising 
in  the  application  of  this  or  that  method,  having  limited  herself  to  the 
most  typical  and  practically  important  cases. 

The  book  consists  of  three  chapters.  In  the  first  chapter  is  given  the 
material  from  linear  algebra  that  is  indispensable  to  what  follows . The 
second  chapter  is  devoted  to  the  numerical  solution  of  systems  of  linear 
equations  and  parallel  questions.  Lastly,  the  third  chapter  contains  a 
description  of  numerical  methods  of  computing  the  latent  roots  and  latent 
vectors  of  a matrix. 

For  the  interest  manifested  in  the  manuscript,  and  for  a number  of 
valuable  suggestions,  I express  my  sincere  thanks  to  A,  I,  Brudno  and 


Go  P,  Akilov, 


April  I,  19f?2 


CHAPTER  1 

BASIC  MATERIAL  FROM  LINEAR  ALGEBRA” 

This  chapter  will  be  of  an  introductory  nature <,  Without  detailed 
proofs , it  will  impart  material  from  linear  algebra  that  will  be  indispen- 
sable to  an  understanding  of  the  following  chapters  « 


§1.  MATRICES 

Xc  An  aggregate  of  numbers— which  are,  generally  speaking,  complex- 
arranged  in  the  form  of  a rectangular  table,  is  called  a rectangular  matrix. 
This  array  will  have  m rows  and  n columns,  and  may  be  set  forth  in  the  forms 


(1) 


A 


the  first  subscript,  then,  designating  the  row,  the  second  designating  the 
column,  in  which  the  element  in  point  is  located. 

This  may  be  abbreviated  to  the  form; 

A - (a±j)  (i  s 1,  2,  •««  9 m • j a 1,  2,  , n)  e 

T’his  translation  was  carried  out  under  the  auspices  of  the  Depart- 
ment  of  Mathematics,  University  of  California,  Los  Angeles.  It  was  edited 
by  George  E.  Forsythe,  National  Bureau  of  Standards,  Los  Angeles. 


— d,  “ 

Two  matrices  are  equal  if  their  corresponding  elements  are  equal „ 

Matrices  composed  of  a single  row  are  called  simply  rows  (or,  as  we 
shall  approach  them  later,  row  vectors) » Matrices  composed,  of  a single 
column  are  called  columns  (or  column  vectors) „ 

If  the  number  of  rows  of  a matrix  equals  the  number,  n,  of  columns,  it  is 
called  square,  and  of  the  n-th  order , 

Among  square  matrices,  an  important  role  is  plsyed  by  diagonal  matrices, 
i«es,  matrices  of  which  only  the  elements  along  the  principal,  (leading) 
diagonal  are  different  from  zero: 


If  all  the  numbers  ©<„  of  such  a matrix,  are  eoual  to  each  other,  the 

i * 3 

matrix  is  said  to  be  scalar: 


1 

and,  if  oc  ® 1,  the  matrix  is  said  to  be  the  unit  matrix: 


Editor 5 s notes 


Faddeeva  and  other  continental  mathematicians  use  the 


symbol  E for  a unit  matrix 


ua  «aj 


00 


Lastly,  a matrix  all  of  whose  elements  are  equal  to  zero  is  called  a 
null  matrix,  or  zero  matrix.  We  shall  designate  it  by  the  symbol  0. 

The  determinant  whose  elements  are  the  elements  of  a square  matrix 
(without  disarrangement),  is  said  to  be  the  determinant  of  that  matrix,  and 
we  write  the  determinant  of  the  matrix  A as  |A|,  or  often  as  d(A) . 


2.  Multiplication  of  a matrix  by  a number.  The  addition  of  matrices . 
A matrix  whose  elements  are  obtained  by  multiplying  all  the  elements  of  the 
matrix  A by  a number  ^ is  called  the  pr oduct  of  the  number  and  the 
matrix  A: 


A matrix  G whose  elements  are  the  sums  of  the  corresponding  elements 
of  A and  B,  matrices  having  like  numbers  of  rows  and  columns,  is  called 


the  sum  of  A and  Bs 


(:.  ) A + -O  ~ 


- h - 

/ aH  + bl 1 

+ ha 

©*o 

^ 4>  h 

in.  n \ 

1 

i + b-,= 

i c.  L cL 

cL^  0( 
CC 

+ b22 

1 

cn  t,n  i 

# © 


a + b . a ~ + b « • ® • 

ml  ml  m2  m2 


a + b 
mn 


mn 


1 e • pe rations  introduced  above  have  the  following  properties,  as  will 

be  readily  seen: 


1.  A + (B  + C)  - (A  + B)  + C » 
2 o A + B ® B + A . 

3.  A + 0 - A. 

Iu  + /3)A  s o<  a + A . 

5.  «<(A  + B)  s A + °<B. 


B::  - A,  B,  and  C are  matrices!  and  (3  are  numbers — generally  speaking, 

complex* 

• The  mill tiplic ati on  of  matrices  „ Multiplic at. ion  of  the  matrices  A 

and  B is  defined  only  on  the  assumption  that  the  number  of  columns  of 

*1 

matrix  A equals  the  number  of  rows  of  matrix”  B»  On  this  assumption,  the 
elements  of  the  product,  C - AB,  are  defined  in  the  following  manner;  the 
element  in  the  i-th  row  and  the  j-th  column  of  the  matrix  C is  equal  to  the 

^Translator’s  notes  Indicating  by  superior  letters  the  dimensions  of 

the  matrix  A,  i8ee,  the  number,  m,  of  rows,  and  the  number,  n,  of  columns, 

mxn 

thus % A the  product  condition  may  be  given  a more  lucid  expression 

mom  pxr 

notat ic a ally * Thus  A ° B is  possible  only  if  p « n*  A and  B are  then 

conformal ■■  .le . 


- s - 


sum  of  the  products  of  the  elements  of  the  i-th  row  of  the  matrix  A by  the 
corresponding  elements  of  the  i-th  column  of  matrix^  Thus; 


:ia  *12  • • * *i  ^ / b, , b,  * • • • b 


i n>  / 12 


(7)  AB  - 


\ I u 

*21  a22  '■*  a2n  1 ! b21  °22  b2 


o o o 


ip\ 

2p 


\ aml  am2 


3tm  \bnl  0 n2  ~npf 


/£ 


°11  CX2  clp 


“21  c22  **8  C2p 


ft  « • • 


c « •••  c 
ml  m2  mp 


where 


(8) 


n 

e.  , 52  a.,  b.  . + s>  b,~ , + *••+•  a.  b . = 2 a.,  b,  . 

2-0  il  Id  i2  in  rg  k=1  lk  kj 


(i  ■‘j  ■ j • * * , ni  * y '2  , • « • , 


It  is  to  be  noted  that  the  product  of  two  rectangular  matrices  is  again 

a rectangular  matrix , the  number  of  rows  of  which  is  equal  to  the  number  of 

rows  (if  the  first  matrix,  and  the  number  of  columns  of  which  is  equal  to  the 

mem  nxp  rroq? 

number  of  columns  of  the  second  matrix?  A • B s C . So,  for  instance, 
the  product  of  a square  matrix  and  a matrix  composed  of  one  column  is  a matrix 
of  one  column o 

The  commutative  law  for  multiplication  does  not,  generally  speaking, 
hold.  We  shall  make  a few  observations  on  this  subject,  however.  The 

1 

Translator* s note;  The  bare  definition  of  matrix  multiplication  given 
in  the  text  provides  a basis  for  the  logically  succeeding  development.  It 
may,  however,  appear  arbitrary  to  the  student  who  has  not  previously  recon- 
noitred the  ground.  For  a brief  account  of  the  source  of  this  definition  of 
matrix  multiplication  see,  e.g.,  [1],  §1-6 „ Abundant  numerical  illustra- 

tions will  be  found  in  [2],  §leIu 


- 6 - 


matrices  AB  and  BA  make  sense  simultaneously  only  if  the  number  of  rows  of 
the  first  matrix  is  equal  to  the  number  of  columns  of  the  second,  and  the 
number  of  columns  of  the  first  is  equal  to  the  number  of  rows  of  the  second. 
Given  the  fulfillment  of  these  conditions,  the  matrices  AB  and  BA  will  both 
be  square,  but  of  different  orders,  unless  A and  B be  square.  Thus  even  to 
put  the  question  of  the  equality  of  the  matrices  AB  and  BA  makes  sense  only 
for  square  matrices.  But  even  in  this  case,  generally  speaking,  AB  ^ BA. 

In  particular  cases  multiplication  may  be  commutative,  and  in  such 
cases  the  matrices  are  said  to  commute . liras,  for  example,  scalar  matrices 
commute  with  any  square  matrix  of  the  same  order,  for 


Hence  follows  the  special  role  of  the  unit  matrix  in  the  multiplica- 
tion of  matrices, to  wits  amongst  all  square  matrices  of  the  same  order, 
the  unit  matrix  plays  the  same  role  as  the  number  one  does  among  numbers. 


(9) 


33 


Indeed, 

AI  “ XA  - A . 

It  can  be  shown  that  the  multiplication  of  matrices  is  associative, 
vize?  if  AB  and  (AB)C  make  sense,  so  also  do  BC  and  Ache),  and 

1.  A(BC)  - (AB)C  e 

The  matrix  product  has  also  these  properties: 

2e  «K(AB)  - (*A)B  - AfaB)  3 
3.  (A  + B)C  " AC  + BC  3 
h * C(A  + B)  « CA  + CB  , 

where  A,  B,  C are  matrices,  ©t  a numb er , 

Let  us  interchange  rows  and  columns  in  the  matrix 


1 

we  obtain  the  transposed  matrix  or  transpose 


'The  word  conjugate  appears  in  the  older  literature 


**11  ®21 


(10)  A1  - A*  - (s 


L.  .)' 


^2  a22 


V 


fin  a2n 


ml 


m2 


a / 

mn/ 


(a..) 


The  following  rule  (the  reversal  rule)  for  a transposed  product  should 
be  noted? 


(10a) 


(AB) ' » B»A' 


In  proof  of  this,  note  that  the  i-th  row  and  j-th  column  of  the 
matrix  (AB) ! is  equal  to  the  element  of  the  j-th  row  and  i=th  column-  of  the 
matrix  AB,  for  this  is  merely  the  interchange  of  row  with  column,  IBe», 
transposition!  and  that  is  equal  to 


(11) 


ajlbli 


a © - 


b . 

i ra 


The  last  expression  is  obviously  equal  to  the  sum  of  the  products  of  the 
elements  of  the  i-th  row  of  the  matrix  Bf  and  the  corresponding  elements 
of  the  j-th  row  of  the  matrix  Af,  i . e « , is  equal  to  the  general  (the  i, 
j-th)  element  of  the  matrix  B!A'0 

In  conclusion  we  shall  remark  that  the  determinant  of  a product  of 
(square)  matrices  is  equal  to  the  product  of  the  determinants  of  the  multi- 
plied matrices % |AB j ® Ja  | jBj,  which  result  is  taken  from  determinant 
theory* 


he  The  partitioning  of  matrices  . Hie  handling  of  matrices  of  high 
orders  requires , as  a ruLes  a large  number  of  operations.  It  is  therefore 
often  expeditious  to  reduce  a computation  involving  matrices  of  high  orders 
to  computations  upon  matrices  of  lower  orders.  Such  a reduction  can  be  ef- 
fected by  partitioning  the  given  matrices i each  matrix  may  be  conceived  as 
composed  of  several  matrices  of  lower  orders,  and  this  subdivision  may  be 
carried  through  in  many  ways,  for  examples 


^11  aX2  al 3 ®iu\  /all  : a12  ®13  ^i  li. 


0-la) 


a21  a22  ^23  ^21 


a31  a32  a33  a3li  / 

/ 


®21;  a22  ®23  a2li 
V31  * a32  a33  a3b 


/®1 1 a12 


a21  a22 


a13  ^ h 


\ 


a a.r 


23  2U 
31  a32  • a33  a"3hj 


The  matrices  into  which  the  given  matrix  is  partitioned  are  called  its 
submat  rices  9 or  cells  . In  such  a partition  the  horizontal  and  vertical 
subdividing  lines  are  of  course  supposed  to  be  carried  across  the  whole 
matrix. 

We  shall  not  concern  ourselves  with  the  general  case  of  the  partition- 

3 

ing  of  a matrix  s but  shall  here  consider  only  a partition  of  square 
matrices  in  which  the  diagonal  submatrices  are  square. 

The  basic  operations  on  partitioned  matrices  whose  diagonal  matrices 
are  of  identical  orders  is  connected  in  a quite  natural  way  with  opera- 
tions upon  the  submatrices  themselves.  It)  wit5  if  we  have 
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(lib) 


and 


(11c) 


where 


(12) 


and 


(13) 


A,,  and  B. . are  square  matrices  of  the  same  order,  then 
li  ii  3 


A + B 


« * • 


$ 
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where 


(13a)  Clj  " AilBlj  + Ai2B2 i + ‘ ’ 


• + A . . B,  . 
ik  kj 


ij>  a = i, 


> k 


We  shall  not  stop  for  a proof  of  the  last  formula,  but  will  note  here  only 
that  the  matrices  A.. ^ and  B.^  can  indeed  be  multiplied,  since  the  number  of 
columns  of  matrix  equals  the  number  of  rows  of  the  matrix  B-^. 

Formulas  (12)  and  (13)  show  that  operations  with  matrices  partitioned 
in  the  manner  indicated  are  to  be  conducted  just  as  if  in  place  of  each 
submatrix  there  was  a number. 

An  important  special  case  of  a partitioned  matrix  is  the  bordered 
matrix.  Having  a square  matrix  A^_.f  of  order  n - Is 


(13b) 


n~l 


all 

al2 

‘“°  h.n-l 

*21 

a22 

a2,n-l 

e © • 

6 0 0 

• a • . 

an-l,l 

an-l,2 

o • • a t 

n-l,n- 

we  form  a square  matrix  of  the  n-th  order,  A , by  appending  to  the  matrix 


A a rows  v , s (a  ,• 
n-1  n-1  hi 


(ain5 


(1U) 


a ^),  a columns  u _ 
^ 5 5 n.n-l  5 n-1 


a , ) and  a number  a = 

n-l,n  5 nn 

^Ln 


n-1 


a =s 
n-l,n 


At  U ,\ 

n-X  n-1 


7n-l  ann 


a a - 8 6 ® a , a 
v nl  n,n-l  nr: 


We  shall  say  that  the  matrix  A has  been  obtained  by  bordering  the 
matrix  A „ Tne.  matrix  A is  naturally  parti  tionab'le. 

Operations  upon  a bordered  matrix:  are  conducted  in  accordance  with  the 
general  rules  for  operations  upon  partitioned  matrices.  Letting 


be  two  bordered  matrices  f rdei  n,  the  meaning  of  m,  v,  u.  a,  and  P.,  x, 

y,  b.  being  the-  £3  lition,  the  following  statements  are  valid: 

r 


«<A 


V,'U  V 
"l 


(11) 


A + B - 


L 


AB  - 


/ M + P 
1 

U + y \ 

\ TT  4*  X 

1 

a + b / 
2 

/ MP  + ux 

My  + ub 

\vP  + ax 

vy  + ab 

2 


Here  M?  and  ux  are  matrices  of  the  (n  - l)-th  order J My.  u'b  are  columns 
composed  of  n - 1 elements!  vF  and  ax  are  analogous  rowsj  and,  lastly, 
vy  ^ ab  is  a number. 
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3s  Quasi-diagonal  matrices  0 Let  us  consider  still  another  particular 
case  of  partitioned  matrices,  namely  the  matrices  called  quasi-diagonal » 
These  are  square  matrices  along  whose  leading  diagonals  are  arrayed  square 
submatrices,  the  remainder  of  the  elements  being  zero.  An  example  would  be 
the  seventh-order  quasi-diagonal  matrix? 


(15a) 


The  cells  of  this  matrix  are  obviously 


(iSb) 


- Ill 


and  the  six  null  submatrices. 

If  two  quasi-diagonal  matrices  are  of  like  structure,  the  product  of 
such  matrices  will  also  be  a qua si “diagonal  matrix  of  the  same  structure, 
the  diagonal  cells  of  which  equal  the  products  of  the  corresponding  cells 
of  the  factor  submatrices. 

The  determinant  of  a quasi-diagonal  matrix  is  equal  to  the  product  of 

the  determinants  of  the  diagonal  cells,  on  the  strength  of  a notable  theorem 

1 

by  Laplace  . 

6.  The  inverse  and  the  adjoint  matrices.  A square  matrix  A - (a..  .) 
is  said  to  be  non- singular  if  its  determinant  is  not  equal  to  zero|  in  the 
contrary  ease  it  is  of  course  singular. 

The  important  concept  of  an  inverse  matrix  is  now  introduced.  A matrix 
B is  called  the  inverse  (or  reciprocal)  of  the  matrix  A if 

(1.6)  AB  “ I e 

We  shall  show  that  the  necessary  and  sufficient  condition  for  the  exis- 
tence of  the  inverse  matrix  is  the  non-singularity  of  the  matrix  A. 

The  necessity  follows  at  once  from  the  theorem  concerning  the  determi- 
nant of  a matrix  product,  for  if  AB  - I,  jA  j jB  j a 1 and  consequently 

|a|  t o. 

Assume  now  that  | A f f 0o  In  order  to  construct  the  inverse  matrix  we 
must  give  preliminary  consideration  to  the  adjoint  (or  ad  jugate)  matrix, 
i.e.,  the  matrix 

"See,  e.g.  [1],  |33. 


(17) 


Here  A. , is  the  algebraic  complement  (cofactor)  of  the  element  a. . in  the 
determinant  of  the  matrix  A,  i 9e.s  is  the  signed  minor  determinant  of  the 
element  a. • . 


We  shall  show  that  the  adjoint  matrix  has  the  following  property; 


(18)  AC  - |A|I  . 

In  demons  tration,  reckoning  the  general  element  of  the  matrix  AC  by  the 
rules  for  matrix  multiplication,  we  find  it  to  equal 


(180 


ailAjl  + *12^2  + 


+ a,  A . , 

in  jn  J 


i.ea,  2 era  for  i f j,  and  |A  j for  i - j,  on  the  strength  of  a familiar 

1 

theorem  on  the  expansion  of  determinants  „ 

The  equality 


(18*) 


CA  - |A|I 


is  established  in  like  manner „ 

The  adjoint  matrix  has  meaning  for  any  square  matrix  A.  From  the 
equality  AC  = jA jl  it  follows  that  the  matrix 

xTr anslatcr ? s notes  An  expansion  in  terms  of  "alien  cofactors"  van- 

« O00  ^ 0 ^ 


ishes  identically 


[1],  §19,  21 


for  non-singular  A,  the  sought  inverse 9 for 


The  constructed  matrix  has  a.lso  the  property 


(20) 


BA  - 1 


which  follows  from  equation  (18?). 

¥e  prove,  lastly,  the  uniqueness  of  the  inverse  matrix.  Assume  that  a 
matrix  X exists  such  that  AX  s I . Multiplying  this  equation  by  B on  the 
left,  we  have  X * B.  If  it  be  assumed  that  Yk  ® I,  a multiplication  on  the 
right  by  B yields  Y = B. 

-1 

The  matrix  Inverse  to  A is  denoted  by  A . It  is  obvious  that 


We  note  that  the  inverse  of  the  product  of  two  matrices  also  displays 
the  reversal  ruler 


(21) 


■•'A  a = A ”i' 

’•  rV  r2  ai 


since 


(21a) 


1?  - 


The  determination  of  the  inverse  matrix  is  one  of  the  fundamental 
problems  of  linear  algebra 0 Equation  (19)  offers  the  possibility  of  com- 
puting the  inverse  matrixj  however , the  computation  of  the  adjoint  matrix 
is  so  labor-consuming  that  the  cited  equation  is  of  importance  only  in 
theoretical  relationships.  Chapter  II  will  be  specially  devoted  to  this 
problem  of  determining  the  inverse  matrix, 

7°  Polynomials  in  a matrix , We  now  define  the  positive  integral 
power  of  a square  matrix,  putting 


In  view  of  the  associative  law,  how  the  parentheses  in  this  product  are 
placed  makes  no  difference,  and  we  therefore  omit  them.  It  is  evident  from 
the  definition  that 


n times 


A -A 


f . n.  m , n+m 


(23) 


Hence  it  follows  that  powers  of  the  same  matrix  are  commutative «, 
We  further  put,  by  definition. 


(23  a) 


An  expression  of  the  form 


(23b) 


A + «,A' 


n 


Ur-l 


o 


18 


where  5 «=<n  are  complex  numbers,  is  called  a polynomial  in  a 

matrix , or  matrix  polynomial  a This  matrix  polynomial  may  be  regarded  as 
the  result  of  replacing  the  variable  X in  an  algebraic  polynomial 

(2b)  Cp  (A)  “ e\  Xn  + es<_  A.  rr““  + « 

7 o 1 n 

by  the  matrix  A. 

It  is  important  to  note  that  the  rules  for  operation  upon  matrix  poly™ 
nomials  do  not  differ  from  the  rules  for  operation  upon  algebraic  polynomials, 
viz  .s 


(2S) 


r . 

given 

f(A)  - Y(A)  ± yfr) 
«(A)  - V(A)X(A)  , 

. 

1 

i then 

f(A)  - \|/(a)  i x(A) 


This  follows  from  the  commutativity  of  the  powers  of  a matrix. 

3„  ^he  characteristic  polynomial. „ The  C ayley-Hamil ton  theorem. 
minimum  pol3momial  „ The  equation 


Tie 


(26) 


I 


a21 


£LC 


- X 


o o © o © © 


a 


nu 


a 


O © © 


a. 


In 


e . o 


a-. 

2n 


nn 


0 
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is  called  the  charac t eris tic  equation  of  the  matrix  A - (a.  .^) » The  left 
member  of  this  equation,  'which  may  be  written  in  the  abbreviated  form 
jA  - A.  I j 3 bears  the  name  oh ar ac teri s tic  polynomial  (or  characteristic 
function)  of  the  matrix , Characteristic  equations  are  frequently  encountered 
in  applied  mathematics^. 

The  direct  computation  of  the  characteristic  function  presents  consid- 
erable technical  difficulties.  If 

(27)  (f  (X)  - |A  - Al|  - (-l)n[Xn  - pf-""1  - pi1"-2  - •••  - pn]  , 

then 


(?-8) 


p_  3 sl  + a co  + 6 • • *■  a 

1 11  nn 

pn  38  (-I) n*”1  |a  | | 


and  the  remaining  coefficients  are  the  sums,  taken  with  the  sign  (-1)^  \ 

of  all  the  principal  minors  of  the  determinant  of  matrix  A of  order  k,  iee#, 

2 

of  the  minors  involving  the  principal,  diagonal  , The  number  of  such  minors 
equals  the  number  of  combinations  of  n things  taken  k at  a time  , 

The  roots  of  the  characteristic  equation  are  called  the  latent  roots 
(characteristic  numbers,  proper  values , eigenvalues)  of  the  matrix  A,  From 
the  well-known  theorem  of  Vi eta  giving  the  connection  between  the  roots  of 
an  equation  and  its  coefficients,  we  have 

^ See,  e,g05  [2],  13,6, 

ISee,  e„g,s  [X],  §37« 
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A- 


(29) 


. 


n .il  i2 


• (-l)^1  p 
n 


The  quantity  p-  - a-,-,  - a.-,,  - • + a is  called  the  trace  (or  spur) 

of  the  matrix  A,  and  is  denoted  by  tr  A. 

Practically  convent  r.t  methods  for  determining  tl  : coefficien  d 
roots  of  the  characteristic  equation  will  be  elaborated  in  Chapter  III,  X'jhich 
will  be  specially  devoted  to  that  group  of  questions.  For  the  moment  we 
leave  them  aside „ 

For  ary  square  matrix  the  following  remarkable  relation,  known  as  the 
C ayley-Kavnil ton  Theorem , ob tai ns ; if  Cf?  (A)  is  the  c h. ar a c t e ri s t i c polynomi al 
of  the  matrix  A,  then  A)  - 0,  that  is.  speaking  somewhat  conditionally, 
matrix  is  a root  of  it  s own  char  Jxr:r. 

For  proof,  let  us  consider  the  matrix  B,  the  adjoint  of  the  matrix 
A - AI.  Since  each  cofactor  in  the  determinant  jA  - Al|  is  a polynomial 
in  A of  degree  not  exceeding  n - 1,  the  adjoint  matrix  may  be  represented 
as  an  algebraic  polynomial  with  matrix  coefficients'1',  i.e.,  in  the  form 


(29a) 


B =*  B + B 0A  + » * • 4-  B 
n— X Tl-Z  o 


Xr.-l 


where  B B are  certain  matrices  not  dependent  on  . On  the 

n-1’  ? o 

strength  of  the  fundamental  property  of  the  adjoint  matrix,  we  have 


(29b) 


(B  . + B • 

KDn*l  n-2 


B A1*"1)  (A  - Xl)  - |A  - All  I 
o 1 1 


- (-l)ri(An  - Pj  W 


P )I 

.t  ry 


uuard  against  confusion  with  a polynomial  in  a matrix.  See,  e.g., 
[3],  Chap.  Ill,  § U,  p.  21  ff. 


This  equation  is  equivalent  to  the  system  of  equations 


oooooee«0t» 


J ‘i^cwa  * y~' 

Multiplying  these  equatic  * . ■ t.  a",  • res 

pectively,  and  adding,  we  obtain  a null  matrix,  on  the  left  side,  and  on 
the  right. 


30) 


+ A 


B- 


(A) 


Tins  <$(k)  ~ 0,  which  is  what  was  required  to  be  proved. 

The  Cayl ey-Hamil t on  relation  shows  that  for  a given  square  matrix  a 


polynomial  exists  for  which  it  is  a root.  Evidently  such  a polynomial  is 
not  unique  , for  if  \jf  (X)  has  such  a property , so  has  any  polynomial  divi- 
sible by  . The  polynomial  of  lowest  degree  having  this  property 

that  the  matrix  A is  a root  of  it,  is  called  the  lidnimnm  polynomial  of  the 
matrix  A . 


We  shall  prove  that,  the  characteristic  polynomial  is  divisible  by 
the  minimum  polynomial. 

Let  q(  A)  and  r(A)  be  the  quotient  and  remainder  obtained  upon  divid- 
ing the  characteristic  polynomial  by  trie  jfdjdmm  polynomial  "Vjf  (X)  z 
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(30a)  fC&)  -\j/  (A)q(A)  + r(A)  , 

the  degree  of  r(  ')  being  of  course  less  than  the  degree  of  "\|f  (A)  . 
Substituting  A for  A in  this  equation,  we  have 

r(A)  - IfU)  - (A)q(A)  55  0 . 

Thus  the  matrix  A proves  to  be  a "root”  of  the  polynomial  r(A)|  it  thence 
follows  that  r(A)  s 0,  since  othenri.se  ijf  (A)  would  not  be  the  minimum  func- 
tion. Consequently  lj/  (A)  divides  (f  ( A)  . 

9,  Similar  matrices . The  matrix  B is  said  to  be  similar  to  the  matrix 
A if  a non-singular  matrix  C exists  such  that  B = C ^AC . Matrix  B is  said 
to  be  obtained  from  matrix  A by  a similarity  (or  collineatory)  transforma- 
tion. 

The  similarity  transformation  has  the  following  properties? 

(30b)  1.  C~\c  + C“XA0C  + •••  + C’h.C  - C~’1(A1  + A0  + •••  + A )C  . 

X c.  n j.  c,  n 

(31)  2.  C“1A1C  8 C“iA?C  •••  C^A^C  - C^CA^Ao  •••  An)C  . 

In  particular,  (C  ^AC)n  - C ^ AnC . 

Hence s 

3.  f (c“xAC)  - C~'~f(A)C  for  ary  polynomial  f(A) 

From  the  last  property  it  follows  directly  that  similar  matrices  have  the 
same  minimum  function. 
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We  shall  show  that  -similar  matrices  have  also  the  same  characteristic 
function. 

We  have 


(31a) 


|b  - \l|  - |c_1ac  -Xl|  - jC^AC  - Ac“1ic|  « 
- |C I“X |A  - Xl|  |c  I - |A-  XI I . 


10.  Elementary  transformations . It  is  frequently  necessary  to  effect 
the  following  operations  upon  matrices  % 

a)  Multiplication  of  the  elements  of  some  row  by  a number^ 
b?)  Adding  to  the  elements  of  some  row  numbers  proportional  to  the 
elements  of  some  preceding  row. 

b?’)  Adding  to  the  elements  of  some  row  numbers  proportional,  to  the 
elements  of  some  following  row. 

Sometimes  such  transformations  must  be  made  upon  the  columns.  Trans- 
form.' at  ions  of  the  type  indicated  are  called  elementary  t r a ns  f o rma  ti ons  of 
the  matrix. 

Any  elementary  transformation  of  the  rows  is  equivalent  to  a premulti- 
plication of  the  matrix  by  a non-singular  matrix  of  a special  form,  as  will 
readily  be  verified.  Operation  a)  is  equivalent  to  a premultiplication  of 
the  matrix  by  the  matrix 
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(32) 


1 


operation  bT)  is  equivalent  to  a premultiplication  by  the  matrix 


operation  b!!)  is  equivalent  to  a premultiplication  by  the  matrix 


(3U) 


<7% 


« • 

1 


1 
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Operations  a)  , b ! ) and  b:s ) are  performed  on  cc"  •mins  by  using  just  such 
elementary  matrices'1'  in  pos tmultiplic ations  of  tine  matrix  undergoing  trans- 
formation. (33)  now  effecting  b!  ) upon  columns,  and  (3U)  effecting  b!). 
Examp 1 es  of  row  operations: 


(3  Us) 


0 


cA 


0 


f\  0 0 \ fs  b 
(3Ub)  j 0 1 0 j 1 

0 '4  1 J \u  v w 


c 

x y z 


f 3Uc) 


a b c 


\0  0 1/  \ 


/ 


V 


a b c 

x y z 

u + v + °<y  w + z 


Translator’s  note:  Note  that  the  elementary  matrices  may  well  be 

regarded  as  having  been  derived  from  the  unit  matrix  by  just  such  trans- 
formations, a),  b?)  and  hl ) , as  it  is  proposed  to  make  upon  the  rows/ col- 
umns  of  A.  Thus,  for  example,  the  elementary  matrix  of  (3Ub)  effects,  by 
p remult  ip  lie  ati  on,  the  following  transformation  of  Ai  row  3 + ^ row  2, 
and  is  itself  the  result  of  such  an  operation  upon  the  unit  matrix.  By 
postmultiplication  it  effects:  column  2 + column  3,  and  is  this  trans- 


formation of  I. 
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In  future  work  we  will  often  have  to  conduct  transformations  of  types  a)  and 
b!)  upon  matrices „ The  result  of  several  row- tr ansf o rma ti ons  of  this  type 
is  equivalent  to  the  premultiplication  of  the  matrix  by  some  triangular 
matrix,  i„e.,  one  of  the  form 


(35) 


o o © e © © 

l 

\ / 

\lnl  v n2  Tnn/ 


with  non-zero  diagonal  elements  njf±±°  ^n<ieed,  each  separate  transformation 
of  form  a)  or  b')  is  equivalent  to  a premultiplication  by  a triangular  ma- 
trix of  the  type  indicated,  and  the  product  of  two  or  more  triangular 
matrices  of  like  structure  (i<,eM  both,  say,  lower  triangular,  as  these)  is 
again  a like  triangular  matrix* 

It  should  be  further  noted  that  the  result  of  several  column-transfor- 
mations of  the  form  b ? ) and  b" ),  such  that  to  each,  j-th,  column  is  added  a 
multiple,  m.  of  the  elements  of  the  i-th  column  (which  itself  remains  un- 
changed),  is  equivalent  to  postmultiplication  of  the  given  matrix  by  a 
matrix  of  the  forms 


11  o Decomposition  of  matrl as  into  the  product  of  two  triangular 
matrices » Triangular  matrices,  that  is,  matrices  of  the  form 


\ nl  cn2  ' ’ ° cnr>  J 


(no  c . . - 0 , 

no  b. . - 0) 

12 


have  a number  of  convenient  properties.  For  instance,  the  determinant  of  a 
triangular  matrix  equals  the  product  of  the  elements  of  the  principal  diagon- 
al; the  product  of  two  triangular  matrices  of  like  structure  is  again  a 
triangular  matrix  of  the  same  structure;  a non-singular  triangular  matrix  is 
easily  inverted  and  its  inverse  is  of  like  structure,  etc. 

The  following  theorems  are  therefore  of  interest. 

THEOREM.  On  condition  that  the  leading  submatrices  of  the  matrix 


(37a) 


are  non- singular , i.e0 , that 
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A may  be  represented  as  the  product  of  a lower  triangular  matrix  and  an 
upper  triangular  matrix » 

The  proof  will  be  carried  through  by  the  method  of  mathematical  induc- 
tion. 

For  n - 1,  the  statement  is  obvious:  (a-^-j ) ® (b^1)(c^)5  and  one  of 
the  factors  may  be  taken  arbitrarily  e Let  the  theorem  be  true  for  a matrix 
of  the  (n  - l)-th  order.  We  shall  show  it  to  be  true  foi  a matrix  of  the 
n-th  order. 

Partition  the  matrix  A into  a bordered  matrix: 


(37c)  a = 


/all  •••  alr.\ 


*21  *2n 


\Va  " *rmj 


A 


n-1 


v\ 


n-l,n 


,An-l  u 


V 


nr 


‘anl  an2 


in5n~'l  ctnr 


■/ 


we  shall  seek  a.  decomposition  A - CB  of  the  matrix  A into  the  product  of 
two  matrices  B and  C of  the  required  forms,  first  having  partitioned  these 
matrices  into  bordered  form  like  that  of  As 


(37d) 


#n-l 


x c. 


B 


nn 


By  the  rule  for  multiplication  of  partitioned  matrices: 
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(37e) 


whence  we  have 


(3?f)  Cn-iBn-l  “ An~l 

Now  such  triangular1  matrices,  C.  , and  B _ , exist,  by  the  induction  hy- 

pothesis  e Furthermore,  from  the  assumption  that  |An^  j / 0,  it  follows  that 

!Cn~lJ  i 0 and  ^ Oo 

Now  x and  y are  found  by  the  formulas 


(37g) 


y s Cn-lu  > 


x ~ vB 


■1 

n-1  > 


wherewith  they  are  determined  uniquely  in  terms  of  u and  v. 

Thus  it  only  remains  for  us  to  determine  the  diagonal  elements  c and 
^ nn 

from  the  equation 
nn  1 


(37  h) 


j b - a — xy 
nn  nn  nn  J 


The  last  equation  shows  that  one  of  the  diagonal  elements  mqy  be  taken 
arbitrarily . 


30 


as  the  deeompo  sit  ion  of  a matrix  into  the  pry- duct  of  two  triangular 

td.ll  be  unique  only  if  we  prescribe  values  for 
the  diagonal  elements  of  one  of  the  tria.ngu.la-  m atrices «, 

; . ' - . Or : ; : ■ ' • , that  b_. . ~ 1,  i = 13  •••  , n, 

Then 


cnn  ” ann 


W 


and  accordingly  matrix  C will  be  uniquely  determined « 

1 e Lot rlx  notation  f or  a system  of  lane  a. r eriuations  „ 
the  system  of  n linear  equations  in  n unknowns’ 


Let  us  consider 


Q -V  4-  *-> 

^ ^ a»} 


-X~  + « 


-in  n 1 


«f.  £ d 


m-*  -fl  _ 


w V}  f* 


(38) 


' t-  » • • + 2 

! til  1 fie.  2 rat  ; 


Utilizing  the  titic  . matrix  multd.pli cation  ? the  tem  may  be  written 
as  a single  equation  in  matrix  notations 


81  ~ 


or  simply  as 

(38")  Ax  - b , 


A signifying  the  matrix  of  the  coefficients  of  the  system,  b the  column  of 
free  members,  and  x the  column  whose  elements  are  the  unknowns «, 

If  the  matrix  of  the  system.  A,  is  non-singular,  we  obtain  at  once  the 
solution  of  system  (38)  by  premultiplying  (38")  by  A s 

(39)  x « A-1b  - — • Bb  , 

|A| 


B being  the  adjoint  matrix  of  A. 

We  shall  show  that  the  last  formula  is  the  matrix  notation  for  the 
familiar  Cramer’s  Rule-, 


(UO) 


x. 


a 


where  A.  is  the  matrix  that  is  obtained  from  A by  replacing  the  elements 
of  the  i-th  column  by  the  components  of  b. 

Indeed,  the  matrix  equation  (39)  is  equivalent  to  the  n equations 


(UOa) 


x. 


Alibl+A2ib2+ ' • -*Ambn 


jA  | 


i SE  1 0.0  p 

- J J 


Since  the  A,^  are  the  cofactors  of  the  element  in  the  determinant  of 
the  matrix  A,  we  obviously  have 


(UOb) 


Auh  * A2ib2 


A 


:rrL  n 


IA1  I 


which  proves  our  statement o 
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2 o n-EIMENSIONAL  VECTOR  SPACE 

In  what  Is  to  follow  an  import  ant  role  will  be  played  by  the  so-called 
^dimensional  vec  t or  space  R . .A  point  I of  such  a space  is  an  aggregate  of 
n numbers 3 as  a rule  complex,  inite  order  ; 


(1) 


= (*~ 


s ^'-  -y  j 


X is  also  called  an  n-dimensional  vector.  The  numbers  x,  , x0 , , x 

are  called  the  components  of  the  vector.  The  number  n is  called  the  dimen- 

"t 

si on  of  the  space. 

Two  vectors  are  said  to  be  equal  only  if  their  corresponding  components 
are  equal.  Fundamental  Derations  on  vectors  are  defined  as  follows;  if 
X - (xp  x.->,  «•  • , x ) and.  T * (y1 , y~,  * * * , jrJ)  are  two  n-dimensional  vectors 
and  a is  an  arbitrary  complex  number,  we  then  put,  by  definition, 


(2) 


X + Y - (X1  + y13  X2  + y2; 


, x^  + y 


aX  ( ax.,  9 ax.A , ° * a , ax 


The  addition  of  vectors  satisfies  the  commutative  and  associative  laws; 


^Translator ! s note;  These  are  extensions,  by  a natural  generalization, 
of  the  familiar  vectors  and  components  of  vector  analysis,  where,  relative 
to  the  basic  vectors  i,  j,  k,  a vector  X = + agj  41  a^k  has  the  compo- 

nents aj , a„,  a-5 « (See,  e,g0,  [Uj,  §'£,)  Here,  abstracted  from  the  rela- 
tion with  basic  vectors  and  with  more  than  three  ” component s" , and  thus  in 
a " space"  of  more  than  three  '’dimensions”,  the  « vector"  terminology  is  pre- 
served as  appropriate  to  the  forms,  operations,  etc. 
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(2 


a) 


2 + T - Y + X 
(X  + Y)  + Z * X + (Y  + Z) 


The  addition  of  vectors  is  connected  with  multiplication  by  numbers  by  the 


distributive  laws 


(3) 


I) 


ax  + 


(a  + b)X  = aX  + bY  . 


The  validity  of  all  \nese  laws  follows  directly  from  the  definition  of 
the  ope  ra  ti  ons » 

For  vectors  of  an  n-dimensional  space  a scalar  product  is  introduced 
in  accordance  with  the  formula 

(W  (X,  Y)  - 2 x^k  , 

k-1 

2 

where  y,  designates  the  complex  conjugate  of  y,, . 

•ii  XA 

It  is  readily  verified  that  the  scalar  product  has  the  following  pro- 
perties 2 


1 

Cf , the  scalar  (dot  product  of  ordinary  vec tors , for  which  there  is 
a convenient  matrix  notation?  see  [lj,  18.9. 
cSee  [ 5] 5 p.  162,  note. 
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1)  (X,X)  > 0 if  X f 0 I (X,x)  - o if  X - 0 . 

2)  (X,Y)  = (fTx)  . 

3)  (X,  - x2,  y)  - (X1,X)  + (xpT)  . 

U)  CaX,Y)  « a(X,Y)  . 

3'5  (X?71  + Y2)  - (XjYO  + (X,Y2)  c 
6}  (X,aY)  - a(X,Y)  * 


In  additions  \/(x,X)  is  called  the  length  of  the  vector.  In  what  fol- 
lows we  shall  designate  it  by  j jX  J j . 

Besides  the  n-dimerisional  complex  space  introduced  above,  it  is  useful, 
to  consider  also  an  n-dimensional  real  space,  i.e.,  the  aggregate  of  vectors 
with  real  components „ 

In  real  space  the  scalar  product  is  equal  to  the  sum  of  ‘the  products 
of  corresponding  components  of  the  vectors  f the  length  of  a vector1  equals 
the  square  root  of  the  sum  of  the  squares  of  its  components* 

We  shall  most  often  have  to  deal  with  real  n-dime ns i o nal  space,  turn- 
ing to  complex  space  only  as  occasion  requires . 


1,  Linear  dependence.  A vector  T - c-,X.  + c*X.  + 999  + c X is  said 
to  be  a linear  combination  of  the  vectors  X^,  Xg,  9 6 9 

It  is  easily  seen  that  if0  vectors  *9*  , Y^  are  linear  combinations 
of  the  vectors  X-  , 999  , XtT(,  any  linear  combination  y^Y^  + 698  + 


will  also  be  a linear  combination  of  the  vectors  X- 


■1  3 


X 


m 


Vectors 


X 


X 


O 5 


* , X^  are  called  linearly  dependent  if  constants 


cl*  c2? 


c exist,  not  all  zero,  such  that  the  equation 


m 
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(5) 


clxl  + 


c«X 


2 2 


+ e X 
m m 


holds.  If 3 however,  this  equation  holds  only  when  all  the  constants  are 
ecrual  to  zero,  the  vectors  X-,  , Xp , , X are  said  to  he  linearly  indepen- 

dent . 

If  the  vectors  X-,  , * ° ' , are  linearly  dependent,  then  at  least  one 
of  them  will  be  a linear  combination  of  the  rest.  For  if,  for  example, 
c f 0 , we  find  from  (5); 


(6) 


Vi 


© 


THEOREM  1 . If  the  vectors  7.,  , * • • , are  linear  combinations  of  the 
vectors  X-^,  * ° ’ , Xm,  and  k > m,  the  former  set  is  linearly  dependent  0 

The  proof  will  be  carried  through  by  the  method  of  mathematical  induc- 
tion. For  m = 1,  the  theorem  is  obvious.  Let  the  theorem  be  true  on  the 
assumption  that  the  number  of  combined  vectors  be  m - 1.  Under  the  condi- 
tion of  the  theorem,  then. 


(6a) 


Y s,-.  v + 


iA 


o ® « © © © ® 


Yk  " GklXl  + 


ckmX: 


'm 


Two  cases  are  conceivable. 


1. 


H J 


Ail  the  coefficients  •••  , are  equal  to  zero.  Then 

T,  are  in  fact  linear  combinations  of  the  m - 1 vectors 
r 


9 
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X?,  008  , Xffio  On  the  strength  of  the  induction  hype  thesis,  ••  • , Y^ 

will  be  linearly  dependent, 

2 o it  least  one  coefficient  of  will  be  different  from  zero,  With- 
out violating  the  generality  we  may  consider  that  c^j  f 0, 

Let  us  now  consider  the  system  of  vectors 


(6b) 


The  vectors  thus  constructed  are  obviously  linear  combinations  of  the 
vectors  X«,  0,0  , X , and  the  number  of  them  is  k - 1>  m - 1„  On  the 
strength  of  the  induction  hypothesis  they  are  linearly  dependent,  i.e„, 
constants  , , * * • , that  are  not  simultaneously  zero  can  be  found  such 


(6c) 


Y ? + 


* 


Replacing  YI  s 
we  obtain 


Y-(",  by  their  expressions  in  terms  of  Y^, 


(6d) 


■iTi  + hi  + 


+ }',j. 


where 


_21 

Ti 


To 


'kl 


~fy  The  numbers  7^, 


? Tjc  ^ e 


not  simultaneously  equal  to  zero  and  accordingly  Y~  9 
linearly  dependent.  This  proves  Theorem  1, 


, Yk  are 


© © © 


A system  of  linearly  independent  vectors  is  said  to  constitute  a basis 
for  a space  if'  any  vector  of  the  space  is  a linear  combination  of  the  vector 
of  the  system* 

An  example  of  a basis  is  the  set  of  vectors 

E e,  «*  (1,  6S  3 0) 

3 -*• 

. 

(?)  ^ e2  35  (°»  ***  > 

6 e • e e » e 

l «n  ” <°.-  °»  -•  > « » 

for  it  is  obvious  that  for  any  vector  X - (x^  , Xg,  , xn)  we  have 

(7a)  1 ’¥l  + x2e2  + "*  + xnen  * 

This  we  shall  call  the  initial  basis  of  the  space.  Such  a basis  is 

not  the  only  one  possible  — quite  the  contrary;  in  the  choice  of  a basis 

one  may  be  arbitrary  within  wide  limits  * Despite  this , the  number  of 

vectors  forming  a basis  does  not  depend  on  its  selection.  In  proof  of  this, 

let  Y,  , •••  , Y,  and  Z-  , » * ° Z be  two  bases,  and  assume  further  that 
1 3 3 K 1 3 m 3 

k > m . The  vectors  Y~  , ° * • , Y,  are  linear  combinations  of  the  vectors 

l3  3 k 

Z^,  « * ° 9 Z^»  In  the  light  of  Theorem  1,  T, s •••  , are  linearly  depen- 
dent, which  contradicts  the  definition  of  basis.  So  k “ m.  Furthermore, 
since  the  initial  basis  is  constituted  by  n vectors,  any  other  basis  -will 
also  consist  of  n vectors.  The  number  of  vectors  forming  a basis  thus 
coincides  with  the  dimension  of  the  space. 
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Let  11, , - 
a linear-  combination  of  U,  , 


, U form  the  basis  of  a space.  An y rector  X will  then  be 
n 


u 


n 


(8) 


x = • 1U1  + ' 9U0  + + 


2 2 


n n 


The  coefficients  of  this  resolution  uniquely  define  the  rector  X,  for  if 


(8a) 
then  ( ! 


L 

n n 


rr  + 
11 


+ tj 
11  n n ? 


1 


1 

f 

1 


' ) U 4°  ® * • 

11 


4-  ( 


n 


i * 

Sn)Un  - 0,  and  accordingly 


0 

U)  3 


" n 


= 0,  in  view  of  the  linear  independence  of 


the  vectors  IL, 


e . • TJ 

3 3 U—  * 


n 

£ 


The  coefficients  n , 


1'  J v n 
vector  X with  respect  to  the  basis  U^,  • 


are  called  the  coordinates  of  the 


Note  that  the  components 


of  a vector  x^,  •••  9 are  the  coordinates  of  the  vector  X with  respect  to 
the  initial  basis . 

2.  Orthogonal  systems  of  vectors . Two  non-zero  vectors  of  a space  are 
said  to  be  or  thogonal  if  their-  scalar  product  equals  zero.  A system  of 
vectors  is  said  to  be  orthogonal  if  ary  two  vectors  of  the  qystem  are  orthogon- 
al to  one  another.  In  speaking  of  an  orthogonal  system,  we  shall  henceforth 
assume  that  all  the  vectors  of  this  system  are  different  from  zero. 

THEOREM  2 e The  vectors  forming  an  orthogonal  system  are  linearly 
independent. 

Proof.  Let  X^,  * » « 9 X^  be  an  orthogonal  system,  and  let 


e,Xn  + &X,  + 

11  & d 


"V*  “ Pj 

ekxk 


(8b) 


O o * 
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in  view  of  the  properties  of  the  scalar  product  we  haves 


(8c) 


0 ’ (oA 


■ 'iAA5  * 


+ — wv^  + •••  + ck(vxi)  = oinxiii: 


2 

and,  since  j |X.,  |j  > 0,  c;  - 0 for  any  i - 1,  2,  •••  , n.  Thus  the  sole  pos- 
sible values  for  c-,  , c«,  , c in  the  equation  cnX-j  + c0X0  + «•>•  + c X - 0 

are  “ c0  52  •••  * cn  - 0,  iee„ , the  vectors  X^,  Xg5  •••  5 X^  are  linearly 
independent,.  It  thence  follows,  first,  that  the  number  of  vectors  forming 
an  orthogonal  system  does  not  exceed  n,  and,  second,  that  any  orthogonal 
system  of  n vectors  forms  a basis  of  the  space.  Such  a basis  is  called 
orthogonal » If  we  have,  in  addition,  | Jx,  J j 53  1,  the  basis  is  said  to  be 
ortho  normal «,  An  example  of  an  ortho  normal  basis  is  the  initial  basis.  . 

From  any  system  of  linearly  independent  vectors  X„ , 9 ° 9 , 

* * 

possible  to  go  over  to  an  orthogonal  system  cf  vectors  X^,  e 9 * X^.  by  means 
of  the  process  spoken  of  as  orthogonali-zatlon . The  following  theorem 
describes  this  process. 

THEOREM  3.  Let  X, , - ° • , X be  linearly  independent . An  orthogonal 

i ! 

system  of  vectors  X., , ® • • , X may  be  constructed  that  is  connected  wit  .a 
the  original  set  by  the  relations : 


X^,  it  is 


(9) 


xj  - X, 

J X2  3 X2  + ^21Xi 

oa»®©600o©«oa 


X,  - X 
k 1 


-,'L 

-1  k- 


j 


- hO  - 


The  proof  mil  be  carried  through  by  indue ti on „ 

Let  X.J , 9 X,!.  j be  already  constructed  and  different  from  zero*  We 


- t 


X' 


m m 


™ ‘’ll  v ifr^l  m-1 


'noose  the  coefficients  jft)  • ••  , so  that  (X^,  X^)  3 0 for 

1,  9i0  , xa  - 1.  ibis  is  easily  done,  for 


(9%  a) 


K> XP  * (v  ZP  + rjdj.  V • 


Now  (X! j l!)  f ",  since  x!  f 0 by  the  induction  hypothesis,  and.  it  is  ac- 

<j  3 3 


cordinglT  sufficient'  to  take 


(?’b) 


7 


j.  1 •■•’lag  I „ • , X.V-  in  equation  (9*)  by  their  expressions  in 

i-  .’ms  rf  3L  j 0oe  , X^  , we  obtain  finally 


- 5 


I ? 
m 


It  remains  to  be  Droved  that  X‘  f 0 e But  this  is  obvious,  for  other” 

m 5 

vd, s.v  the  vector  X would  be  a linear  combination  of  the  vectors 

m 

X1,  ”co  9 T lf  which  contradicts  the  condition  of  the  theorem.  The  basis 
of  the  induction  exist s,  since  for  m - 1 the  theorem  is  trivial,, 

One  may  pass  from  any  orthogonal  system  to  the  corresponding  ortho- 
normal  system  by  dividing  each  vector  by  its  length. 


la  - 


The  le  1 a permits  of  great  latitude  in  the  choice  of  an 

orfchonorma.1  ha  f • may  pass  from  a ry  basis  to  an  orthonormal  one 
by  • rthogc;  • i h . normalization, 

ri lie  eealee  pv--;.ti  of  two  vectors  is  very  simply  expressible  in  terms 
of  l.:e  - "hi  "V  se  vectors  with  respect  to  any  orthonormal  basis, 

for,  if  h;  ? sbo  „ e an  orthonormal  basis,  and 

y - Vl  + •-  + T(A  -• 


(9'd)  I » tlL  ♦ •••  + J 0 ; 

1 d n n 5 

then 


TJ 

n n 


(98e) 


n n 
2 2 

isX 


k ha 


n 


isl 


T 


a 


Thir  th  : :a  j e 1 of  the  scalar  product  in  terms  of  the  coordinates 
of  the  vectors  mil  r vr  set  to  any  orthonormal  basis  coincides  with  its  ex- 
pression in  terms  of  i/h.  components  of  the  vectors,  i.e.,  in  terms  of  the 
coordinates  with  rosp  f to  the  initial  basis. 

3,  Transfomcf  lea  of  coordinates  . Let  us  elucidate  the  change  in 
the  coordinates  of  a vector  that  accompanies  a change  of  basis. 

Let  e, , e.;,  a •* * , and  e~, , e.,,  • ° * be  two  bases,  and  let 


do: 


^Ll6!  ■ a21e2 


+ 


< 


: 12 el  + a22e2  4 8,0  + an2e 


n 


a © © 


m e«  + a«  © + • • • + a e 

In  1 <dn  2 ns  n 


« ® • 


\\2 


We  connect  'with  the  transformation  of  coordinates  a.  matrix  A,  the 
columns  of  which  consist  of  the  coordinates  of  the  vectors  , e,s,,  °««> 


with  respect  to  the  basis  e.,  , «•<>  , 


e s i . e o , the  matrix 


n 


(11) 


A 


ail  a12 


, aril  an2 


® * ♦ S. 


m 


a21  a22 


o o o o 


mi 


The  matrix  A is  non-singular,  for  it  lias  an  inverse,  by  means  of  which 
the  vectors  e . , , * * “ , are  expressible  in  terns  of  the  vectors 


p Ct  « o « 


J 

1 

h 


Now  designate  by  :x.,  9 • , x the  coordinates  of  a vector  X with  respect 

to  the  basis  e1  , e-,,  *»•  , e.n , and  by  x1  , Xp,  xv  its  coordinates  with 
respedt-  to  the  basis  el , el.  ej  , Let  ns  determine  the  relation  of  depen- 
dence  between  the  old  and  the  new  coordinates*  We  haves 


(11a.) 


n6! 


2Cj@9  + 
c c 


+ x e 


n 


t i , 

+ Xn&Sj  f » 


*lel 


% i 

+ x 
n n 


( allel  + a21e2  + 

! 

“■2VC112"1  ’ “22c2 


+■  a , e ) + 

ill  n 


+ xc(su«e,  + su«e0  + •••  + a -e)  + 


nr  n 


4* 


xi(alnel  + a2ne2  + * Ven>  ' 

Kiri + *iA  *■■■’*  vX + 

(a2l^  4.  ajgx^  4 ...  + a2nx;)e2  + 


© e & oaoeoa©© 

+ (a  x1  + a ,-.xA  + 000  + a x')e 
nl  1 n2  T2  nn  n n 


4s 


easqa 


xv  = x 
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:sb  uaqqT.m  aq  item  (ox)  uoanauaqjq 


"ssoisdsqns  papAxup  sb  uiaqp  puaSau  ppaqs  ajq  ‘uoxpxuxjap 
uno  jo  asuas  aqp  up  saoadsqns  aq  qoaa  ppxu  £a°eds  auxpua  aqp  jo  pasoduioo 
pas  aqp  ospa  sa  ‘uopoaA  ppnu  aqp  jo  Xpapos  pasoduioo  pas  aqp  paqp  apoq 

“aoadsqns  aqp  jo  uoxsuaurrp  aqp  pa  ppa  o sp  uaqiunu  paqp  ° p uiauoaqp 
jo  pq3pp  aqp  up  5uoxpoapas  spp  jo  uauuaui  aqp  uo  puadap  pou  ppm  spsaq 
a Suxuiuoj  suopoaA  jo  uaqmru  aqp  naAauQq;  ‘spseq  a jo  aopoqo  aqp  up  apnp 
-xpap  qonui  sapaoxpup  qpuoj  pas  Supuoseau  aqp  paqp  pajpreuiau  aq  ppxu  pp 

* spseq  uno 

paponupsuoo  aAaq  ppaqs  an  f*a®p  ‘aoadsqns  uno  psnaqxa  suoppaupquioo  uaauxp 
upaqp  paqp  qons  ^x  6 <>«>»  ‘ 4[  suopoaA  jo  uiapsXs  appupj  a paponupsuoo  aAaq 
snqp  ppaqs  af&  “suopoaA  puapuadapixp  iCpuaauxp  u uaqp  auoui  aq  pouiieo  auaqp 
aoads  aqp  up  uoj  ‘jCqqauxuiuapux  uo  o3  pouuao  ssaooud  aqp  8 qpuoj  os  pua 

‘uiaqp  jo  puapuadapup  iCpuaauxq  punoj  aq  ppm  uopoaA  a ‘aoadsqns  aqp  psnaqxa 

p t T 

pou  op  Xaqp  jp  s^X  PTJB  X J°  suoppaupquioo  uaaupp  jo  pas  aqp  uapqsuoQ  “ Lx 

jo  puapuadapup  Xpuaeupx  ‘'punoj  aq  pqpw.  °x  uopoaA  a ‘anup  sp  Zreupuoo  aqp 

jp  8 spsaq  spp  suiuo  j uaqp  "*-x  eaoadsqns  auxpua  aqp  psnaqxa  asaqp  jp  “"^x0 

mu  oj  aqp  jo  suopoaA  ppa  t8a#i  Suoppaupquioo  uaaupp  spp  ppa  uapxsuoo  pua 

‘ouaz  uiouj  puauajjxp  ‘"^x  uopoaA  Xueuppque  ua  psuxj  ajfap  “uauuaui  Sumoppoj 

aqp  up  spsaq  aqp  ponupsuoo  sn  pap  “aoadsqns  auxpua  aqp  psnaqxa  Asm  auo 

qopqM.  jo  suoppaupquioo  uaaupp  i£q  ‘suopoaA  puapuadapup  Xpuaaupp  jo  pas  a sp 

auaqp  paqp  ‘®a“p  ‘aoadsqns  ^uaAa  up  spspxa  spsaq  a paqp  uoqs  ppaqs  ajp 

8 Q ‘ .o.  ‘ lq  suopoaA  jo  uiaps.^3  aqp  Aq  pauuads  aoadsqns  aqp  aq  op  ppas 

sp  uauuaui  spqp  up  apqpponupsuoo  aoadsqns  v “aoadsqns  a apnpxpsuoo  iqsnoxA 

-ci0  TIP*  waqp  jo  suoppaupquoo  uaeupp  apqpssod  ppa  jo  pas  aqp  uaqp  ‘uaAx§ 

aq— puapuadap  jCpuaaupp  ueAS  uo— puapuadapup  uCpuaauxp  aua  paqp  ‘ ...  f 
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1 

qqpi  sapTOUTOO  esraimxoo  qu epuadipuT  ipj'6911!!  jo  jlTqiinH  mmpciw  aqq  qsjb  sb 
‘xpiqBw  b jo  sx-io  j quapaedapiiT  Z^OBaurp  jo  lisqprui  umurpreiu  aqj  oH3H0aHI 

gpTjBA  st  msjoaqq  qtreqjodiuT  Sumopjoj  aqj 

e (J  < HI  ‘XTJJBUl 

X m jBxnSuBqoaa  b jo  c9QUBqsuT  joj  fasBo  aqq  up  sb)  axqesoduioo  qou  ajB 
ao  oaaz  oq  X8^9  9**b  «iaq3Tq  puB  p + j aapao  jo  sjcutui  XT8  qraq  «*9p*io  jo 
jouTiii  cjaz-uou  b s-qspca  eoaqq  xtjjbtjj  aqq  jo  s-jouttii  aqq  3uoure  q-euq  qons  j jaq 
-umn  b st  XTjqeni  b jo  qtrea  aqq  ^spaow,  jaqqo  u.r  *y  xpiqBut  aqq  jo  quBJ  aqq 
pa XXs0  ST  j:9P^o  qsaSjBx  jo  st  jouthi  SurqsTtiBA-iToii  jcaAaqBqm  jo  japan  aqj 

• quamaSuBJJB  TBJnq'eu  it  aqq  irt  * y xrjqBUi 
aqq  jo  sinmpoo  q x&ib  puB  smoj  q £jjb  jo  suq xq o a s j:  aqu  t aqq  jb  paqBnqxs  squaw 
-aqa  aqq  jo  pasodwoo  japjo  qq-q  aqq  jo  quBuxwjaqap  b st  y xruqBw  aqq  jo  q 
aapo;o  jo  joutui  b cXx+oexa  ajopj  0xTuqBW  sxqq  jo  jqutw  b papist  st  xtjjbui  b 
jo  STtiunxoo  pnB  shoj.  am  „q t j sl  sraunpoo  puB  shot  asaqM.  quBuxwuaqap  duy 


« V 


SY  xtjjbui  JT'xr^uBjoaj 
£wa  oq  aq.BTjdo.tddB  s qtrea:  jo  qdaouoo  quBqjodwx  aqq  aonpojquj  Qpi  “xpHriu  b 
jo  qiiBJ  aqq  pm?  aoBdsqns  e jc  uoTS-uawxp  aqq  u©55qal^  uorqoauuob  aqj 
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From  this  theorem  it  follows  directly  that  the  dimension  of  the  sub- 
space  spanned  by  the  vectors  Up  , U,  equals  the  rank  of  the  matrix 
composed  of  the  elements  of  these  vectors. 

In  proof,  if  the  rank  of  a matrix  whose  columns  are  the  components  of 
the  vectors  IL, , 000  5 U_  equals  r,  then  of  these  m vectors  r will  be  linearly 
independent,  and  these  mil  correspond  to  the  linearly  independent  columns  of 
the  matrix,?  all  the  rest  of  the  columns  will  be  linear  combinations  of  them. 
Any  vector  subspace  is  a linear  combination  of  the  vectors  Up  •••  , U , 
which  are  themselves  linear  combinations  of  but  r selected  linearly  indepen- 
dent vectors.  Consequently  any  vector  is  a linear  combination  of  r vectors, 
and  therefore  the  rank  r of  the  matrix  in  question  coincides  with  the  dimen- 
sion of  the  subspace  s 


13.  LINEAR  TRANSFORM  HONS 

l„Let  us  associate  with  each  vector  X of  a space  a certain  vector  Y of 
the  same  space.  Such  an  association  we  shall  call  a transformation  of  the 
space.  We  shall  designate  the  result  of  the  application  of  transformation 
A to  the  vector  X by  AX. 

We  shall  call  the  transformation  A linear  if 

1*  A{©$X)  - ^ AX,  for  any  complex  number  <©<j 
2.  aCx^  + X2)  - AXt_  + AX2. 

We  shall  define,  furthermore,  operations  upon  linear  transformations.  The 
product  of  the  linear  ■ transformations  A and  B,  AB  = C,  will  be  a transfor- 
mation constituted  by  the  transformations  B and  A in  turn,  B being  com- 
pleted first,  and  then  A. 


The  product  of  linear  transformations  is  a linear  transformation,  as 


is  readily  seen,  since 


X, 


+ X2))  s aCB^  + BXg) 


(1) 


+ ABX2  » CX3  * cx,  , 


C«<X  - AB^X  • A«*BX  - ^ABX  « ^ CX 


The  siM  of  the  linear  transformations  A and  B will  be  a transformation 
C which  associates  the  vector  X with  the  vector  AX  + BX»  This  sum  of 
linear  transformations  is  obviously  itself  a linear  transformation. 

2e  Representation  of  a linear  transforation  by  a matrix.  Let  us 
choose,  in  the  space  R , some  basis  e^,  e-1}  ee*  , ©r . A linear  transfor- 
mation relates  to  the  vectors  of  the  basis  the  vectors  Ae„ , Ae~,  *»»  , Ae  8 
Let  Ae ...(  , ° • e , Ae  be  given  in  terms  of  their  coordinates  with  respect 
to  the  basis  e^,  e0,  e^,  i,e6,  let 


j Ae1 


el  + a21e2  * + anlen 


(2) 


Ae0  • ai2el  + a22e2  + s°°  + a-^e 


e © & © e o o 


n2  n 


o o e 


Ae 


n 


^n6!  + *2**2  + * annen 


Consider  the  matrix  A,  its  columns  composed  of  the  coordinates  of 

the  vectors  Ae, , Ae0,  ® • • , Ae  % 

==  ± = d 7 — n 
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(3) 


TL L 


a 


n2 


We  shall  show  that  the  matrix  A uniquely  defines  the  linear  transforma- 
tion1 * * * * „ 

Indeed,  if  the  matrix  A Is  known  for  the  linear  transformation,  I.ee, 

if  Ae„  , Ae«,  c ° 6 ° Ae  are  determined,  this  is  sufficient  to  find  the  transfor- 
” l9  — 2'  — n 9 

mation  of  a ny  vector,  for  if 

(3a)  X “ x1e1  + + x er  , 


then 


(3b) 


AX 


x,  Ae 


1 


■+■  x 


. Ae 
n-  n 


Hent  the  coordinates  of  the  transformed  vector  are  easily  found,  for 
we  have 


(3c) 


AX 


n 


k-1 


n 


k k 


x.Ae. 

i—  i 


n n 

2 2 a.  oX.e, 

. . , _ ki  i k 
i"l  k“l 


whence 

(3d)  5'k  ' A VXi  » 

1-1 

1 

Author ? s remarks 

Mote,  however,  that  the  matrix  of  the  coefficients  in  the  relations 

(2)  forms  a matrix  which  is  the  transpose  of  that  that  we  connect  with  the 


linear  transformation 


$1 


or,  in  matrix  notation. 


(U) 


y “ Ax 


where  y and  x are  columns  of  the  coordinates  of  vectors  Y and  I. 

Conversely,  an  arbitrary  matrix  A may  be  connected  with  a certain 
linear  transformation „ Indeed,  the  transformation  given  by  the  formula 


where  y and  x are,  as  above,  the  columns  of  coordinates  of  the  vectors  Y and 
X,  is  linear  for  any  matrix  A. 

The  established  one-to-one  correspondence  between  transformations  and 
matrices  is  preserved  when  operations  are  performed  upon  transformations, 
for  the  matrix  of  the  sum  of  transf o mat-ions  equals  the  sum  of  the  matrices 
of  the  summand  transf o rtna ti ons , and  the  matrix  of  a product  of  transforma- 
tions equals  the  product  of  the  matrices  corresponding  to  the  factor  trans- 
formations, 

3 « jhe  connection  between  the  matrices  of  a linear  transf  orma ti  on  with 
respect  to  different  bases  9 We  will  now  elucidate  how  the  matrix  of  a 
linear  transformation  changes  with  a change  of  the  basis  of  the  space. 

Assume  that  from  the  basis  e_ , * « ° , e we  have  passed  to  the  basis 
e* , * • • „ e?  , and  let 


(Ua) 


y s Ax  , 


1 


’ll6!  + c2Xe2 


(lib) 


© 


9 


© 


O 
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that  is* 

E - C E 

The  coordinates  of  any  vector  of  the  space  will  have  changed  accord- 
ingly by  the  formula 

(lie)  x **  Cx"  * 


where 


The  matrix  of  the  transition*  C,  is  evidently  non-singular.  It  -trill 

coincide  with  the  matrix  of  the  linear  transformation  sending  the  basis 

e_  , e„,  •••  , e into  the  basis  e*  el,  , el 
l ? 2 5 J n l}  2’  9 n 

Let  us  low  consider  a linear  transformation  A.  and  let  the  matrix  A 
correspond  to  it  with  respect  to  the  basis  e,*  e0*  •••  * e * and  the  matrix 

ft  f 

3 with  respect  to  the  basis  e.  , e„,  , e „ 

i ’ d3  3 n 

If  x is  the  column  of  the  coordinates  of  the  vector  X with  respect  to 
the  basis  e,^ , •••  * , and  x?— -with  respect  to  the  basis  en  , •»*  * e^, 

y and  y!  being  the  analogous  columns  for  vector  Y , we  have 


(he) 


Y m Ax 
y?  - Bx! 
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But  x s Cx5,  y - Cy8,  and  therefore 
(Ilf)  Cy!  ~ ACx* 
or 

(Ug)  y*  - Bxs  » C “ACx" 


Thus  similar  matrices  correspond  to  the  same  linear  transformation  with, 
respect  to  different  bases.  Furthermore,  the  matrix  by  means  of  which  the 
similarity  transformation  is  effected  coincides  with  the  matrix  of  trans- 
formation of  coordinates, 

iu  The  transfer  rule  for  a matrix  in  a scalar  product.  Let  X and  Y 
be  two  vectors  given  by  their  components  with  respect  to  the  initial  basis: 

X = (Xp  0 8 0 ? xv-,)s  Y s (y,  , 5 Yn)3  and  let  A be  a linear  transforma- 

tion with  matrix  A 3 (a,^) . Designate  by  A‘  the  linear  transformation  with 
matrix  A" 9 the  elements  of  which  are  the  complex-conjugates  of  their  counter— 
parts  in  A,  and  which  are  placed  in  transposed  position:  A'  - 


* OOVE3UB  T t y -x=n=ssx=>  . 

) s (,  . ) , We  shall  prove  the  following  formulas 
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(AX,  Y)  3 (X,  A*Y) 


In  demonstration,  we  have 
(5a)  (AX  3 Y) 


n n 
2 2 

i-1  k“l 


kV: 


1 .Z.  \ * Vi  * (X’ 

K3±  1”1 


5„  The  rank  of  a linear  transformation e Let  A be  a certain  linear 

transformations  The  set  of  vectors  AX  trill  obviously  constitute  a sub  space , 

which  we  shall  denote  by  AH  * 

— n 

The  dimension  of  this  subspace  is  said  to  be  the  rank  of  the  transfor- 
mation A, 

We  shall  show  the  rank  of  a transformation  to  be  equal  to  the  rank  of 
the  matrix  corresponding  to  this  transformation  on  any  basis  whatever, 
e^s  e,; , •••  , en„  Obviously  the  subspace  AR^  is  spanned  by  the  vectors 
A'e^,  Ae,. , t Aer  „ The  dimension  of  ARv.  is  accordingly  equal  to  the  rank 
of  a matrix  whose  columns  are  composed  of  the  coordinates  of  the  vectors 
jUi,  AeOJ  •»«  3 Ae,f,_»  i.eCJ  to  the  rank  of  a matrix  corresponding  to  the  trans- 
formation . 

Since  the  dimension  of  a subspace  does  not  depend  upon  the  selection  of 
the  basis,  it  follows  from  the  foregoing  that  the  ranks  of  similar  matrices 
are  equal,, 

6„  The  latent  vectors  of  a linear  transformation.,  By  a latent  vector 
(characteristic  vector,  proper  vector  or  eigenvector)  of  a linear  transfor- 
mation A is  meant  any  non-zero  vector  X such  that 

(6)  ax  ■ Ax  , 

where  X is  any  complex  number « 

The  number  is  called  the  latent  root  ( char ac teris tic  value 3 proper 
value , or  eigenvalue)  of  the  transformation.  The  spectrum  of  the  trans- 
formation is  the  aggregate  of  its  latent  roots • 


. 


The  latent  roots  and  latent  vectors  of  the  transformation  may  be 
determined  in  the  following  maimer,.  Let  the  transformation  A be  connected 


•with  the  matrix  A - (a^)  with  respect  to  some  basisj  let  the  coordinates 


x 


of  the  latent  vector  X3  with  respect  to  this  basis } be  e»°  , _ 
The  coordinates  of  the  vector  AX  will  then  bes 


(6a) 


Tl 

auA 

,k<L 

\ 


n 


* 9 « - 2 a , x, 


\ 


* k=l  h*  ^ 1 


and  thus  for  the  determination  of  x,,  , x and  the  latent  root 

l.5  d 5 n 

we  will  have  the  system  of  equations; 


X 


*11*1  + “12*2  + 800  + Vn  s Axl 


(7) 


a^Xj  + + 080  + 


a 


ZrT 


~ Ax, 


t)  e o © © 


E 2C-,  x 3.  A + 
ruL  j»  Hit  d> 


o e o o 


c + a x 
nn  n 


A 


x 


n 


(a 


'll 


x,  + a, + 000  + ajn  x " 0 


(7s) 


211  ^ 2£ 


) x,  + •••  + a0„  xv  * 0 
^ dxi  n 


© e o 


o e o e o o 


© o 


a _ X-,  a eytLtr*  ^ 6 

nl  jl  n2  d 


m 


x * 0 
n 


This  system  of  homogeneous  equations  in  x^s  s x^  will  have  a non-zero 
solution  only  in  case 


i.e.,  if  a is  a root  of  the  characteristic  polynomial  of  the  matrix.  Thus 
the  following  is  valid % 

THEOREM*  'The  latent  roots  of  a transf  orm aid  on  coincide  with  the  roots 
of  fhe  characteristic  polynomial  of  the  matrix  th at  is  connected  -with  this 
transf orma tl ,.n  with  respect  to  an  arbitrary  basis . 

From  the  theorem  often  called  the  fundame  il  theorem  of  higher  algebra, 
we  know  that  every  polynomial  has  at  least  one  root*  A linear  transforma- 
tion will  c:  .‘.sequent ly  have  at  least  one  latent  root,  which  may  be  complex 
even  though  the  matrix  of  the  transformation  be  real*  In  view  of  the  theory 
of  linear  homogeneous  systems  of  equations,  there  will  be  a non-zero  solu- 
tion of  system  (7)  for  each  latent  root,  i.e.,  with  each  latent  root  at 
least  one  latent  vector  is  associated, 

Obviously  if  X is  a latent  vector  of  the  transformation  A,  then,  for 
all  c / 0,  cX  will  also  be  a latent  vector  of  transformation  A correspond- 
ing to  the  same  latent  root.  Furthermore  if  several  latent  vectors  corres- 
pond to  seme  one  latent  root,  then  any  linear  combination  of  them  will  be  a 
latent  vector  of  the  transformation  associated  with  the  same  root*  The  set 
of  latent  vectors  corresponding  to  a single  latent  root  forms  a linear 


- £7  - 

s lib  space  „ We  shall  establish  that  its  dimension,  %,  dees  not  exceed  the 
multiplicity  of  the  latent  root*  Indeed,,  let  X,  , ”oe  , 1j  be  linearly 
independent  latent  vectors  corresponding  to  the  single  latent  root  A « 
Construct  a basis  of  the  space 


000  , X , having  taken  as  the  first  % 


vectors  the  vectors  X,  , • ° ° , Xy*  With  respect  to  this  basis  the  linear 

J-  ' JL 

transformation  under  consideration  is  connected  with  a matrix  whose  first 
columns  have  the  form 


i =,  0 o » !>  0 

"X 


(7'b) 


'1 


© o o o © o 


0 0 


e © ^ ■„ 


e>  9 © 


0 


5 AXy  - Xx  X. 


Now  (X  - i 


is  a factor  of  the 


for  AX,  - ° 

characteristic  polynomial  of  this  matrix,  and  accordingly  A.,  is  of  multi- 
plied iy  k not  less  than  %9  i0e«?  % ^ k«  It  would  naturally  be  supposed 
that  % a k,  i„e<,3  that  to  a k -multiple  root  of  the  characteristic  polynomial 
there  correspond  k linearly  independent  latent  vectors „ But  this  is  in 
fact  not  true*  In  reality , the  number  of  linearly  independent  vectors  may 
be  less  than  the  multiplicity  of  the  latent  root® 

Let  us  confirm  the  preceding  statement  with  an.  example e Consider 


the  linear  transformation  with  the  matrix 
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(7«e) 


/ 


■3.  / 
/ 


Then  ]A  - Al|  - (/'  - 3)" , and  thus  A.  3 3 is  a double  root  of  the  charac' 


teristio  polynomial . 

The  system  of  equations  for  determining  the  coordinates  of  the  latent 


vector  of  the  transformation  A will  be? 


3x-  + x,  “ 3 x. 

i jl 

(7’d) 

3x2  s 3x9 


whence  x~,  - 0,  and  thus  all  the  latent  roots  of  the  transformation  in  ques- 
t-ion  will,  be  (x^,  0)  = 3C^  (1,  0) „ So  in  this  instance  only  one  linearly  inde- 
pendent- vector  is  associated  with  a double  root. 

Generally  speaking,  the  coordinates  of  a latent  vector  on  the  chosen 
basis  are  to  be  determined  from  the  system  (7)  of  linear  equations,  in 
which  for  X the  latent  root  A is  substituted*  But  as  is  known  from  the 
theory-  of  systems  of  linear  equations,  the  number  of  linearly  independent 
solutions  of  a homogeneous  system  equals  n - r,  where  r is  the  rank  of  the 
matrix  composed  of  the  coefficients  of  the  system.  Therefore  if  r denote 
the  rani-;  of  the  matrix  A - A I,  % - n - re  Thus  n - r - k,  and  the 
equality  does  not  always  hold. 


In  case  the  basis  does  not  change  in  the  course  of  the  argument,  we 


shall  often  identify  the  linear  transformation  with  the  matrix  of  the 
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linear  transformation  with  respect  to  tills  basis,  and  any  vector  space 
•with  the  columns  of  its  coordinates. 

On  this  agreement,  it  makes  sense  to  speak  f a latent  -rector  of  a 
matrix  9 understanding  by  this  a column  x satisfying  the  condition 

( 7 ? e ) Ax  ® A x „ 

We  remark  that  if  a latent  root  of  a real  matrix  is  complex,  the  co- 
ordinates of  an  associated  latent  vector  will  be  complex,,  A vector  whose 
coordinates  are  the  complex-conjugates  of  those  cf  a given  latent  vector 
of  a real  matrix,  is  also  a latent  vector  of  that  matrix,  and  is  associated 
with  the  co mpl ex- conjugate  latent  root.  To  convince  oneself  of  this,  il- 
ls enough  'to  change  all  numbers  in  the  equation  Ax  " Ax  into  their  complex 
conjugates , 

7 » Properties  of  the  latent  roots  and  vectors  of  a matrix , We  shall 
establish  several,  properties  of  the  latent  roots  and  vectors  of  a real 
matrix. 

First  of  all  we  note  that  a matrix  and  its  transpose  have  identical 
characteristic  polynomials  and  consequently  identical  spectra.  This  is 
evident  since  \k"  - Alj  ~ jA  - AI  j5  on  the  strength  of  the  fact  that  a 
determinant  is  not  altered  when  its  rows  and  columns  are  interchanged,, 

Now  let  and  A denote  distinct  latent  roots  of  the  real  matrix 
A,  and  A „ the  complex  conjugate  of  A , As  we  saw  above,  A„  is  also  a 

i S 3 

latent  root  of  matrix  A,  and  thus  also  of  the  transposed  matrix  V „ Let 
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A be  a veeio'i  - the  nix  A belonging  to  the  latent  root  A^, 

and  X*  the  latent  vector  of  the  matrix  A-  belonging  to  the  latent  root  A 
s s 

¥e  shall  show  that  I and  X are  orthogonal . 

I*  g; 

? . 

With  this  object  in  view,  let  ns  form  the  sea',  ar  produce  (AX  , X ) 

3.  S 

and  reckon  it  by  two  methods. 

By  one  method  the  reckoning  is 


(7>f) 


(AX  , x’)  - 0.pxr,  ip  - >r(xr,  x’) 


* 


On  the  other  hand,  since  matrix  A is  real,  we  have  A 35  A?,  and  therefore 


,7’g) 


(ax  , xl)  - (X, 


A *X  * ) 


- (x  , A xl)  " Ajx  . Xl) 


'Thus  A (X  , X '}  ~ ■ (X  . X " } . But  the  condition  was  that  A A A „ and 

r r3  s s r3  s v s? 

therefore  (X„,,  X’,)  * 0,  which  is  what  was  required  to  be  proved* 

T'  s 

In  case  all  latent  roots  are  distinct,  the  demonstrated  property  gives 

2 

i - n relations  of  orthogonality  between  the  latent  (rectors  of  matrices  A 
and  A5*  We  shall  later  return  to  these  properties  in  more  detail,* 


Foi  a real  symmetric  matrix,  the  properties  of  orthogonality  are  con- 
siderably simplified  thanks  to  the  fact  that  all  its  latent  roots  are  real. 
In  proof,  letting  A and  X be  respectively  latent  root  and  vector,  we  have 
(AX,  X)  - A(X,  X)j  (AX,  X)  - (X,  ATX)  - (X,  AX)  - (X,  Ax)  - X (X,  X), 

Thus  (A  - A)(X,  X)  * 0,  and  as  (X,  X)  > C9  X ~ X , i.e.,  A is  real. 

From  the  reality  of  the  latent  roots  of  a real  symmetric  matrix  it  fol- 
lows that  vectors  with  real  components  may  be  taken  as  the  latent  vectors 
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belonging to  those  roots;  the  components  mil  indeed  be  found  by  solving  the 
linear  homogeneous  system  with  real  coefficients . 

The  orthogonality  property  of  the  latent  vectors  of  a real  symmetric 
matrix  is  very  simply  formulated  in  view  of  the  coincidence  of  the  matrix 
with  its  transpose  and  the  reality  of  the  latent  roots,  viz . 2 latent  vec- 
tors belonging  to  dis tinct  latent  roots  are  orthogonal . 

8.  The  latent  roots  of  a positive  definite  quadratic  form.  A homo- 
geneous polynomial  of  the  second  degree  in  several  variables  i,,  •••  , x is 
called  a quadratic  form . We  shall  consider  only  those  with  real  coefficients. 
Ary  quadratic  form  may  be  written  as 


(7?h) 


n. 


£ W>  x2>  "•  > xn)  ' . -h  aikxixk  • 

i.k=l 


where  a,.  « 


A quadratic  form  is  said  to  be  positive  definite  if  its  values  are 
positive  for  any  real  values  of  x,,  9 no^  all  zero. 

It  is  evident  that  the  diagonal  coefficients  of  a positive  definite 
form  are  positive,  for 


(7*i) 


^11  IKI3  ***  9 ^3  a22  ” 1 


a 


nn 


22 

(0,  0 


. ...  ( 1 ) 

9 •‘-9  3 •'  5 


3 ^3 


1 

3 * 


Denoting  by  X the  vector  with  components  (x^,  , xn)  3 we  "write 

a quadratic  form  as  ( AX,  X) , A.  being  the  matrix  composed  of  the  coefficients 


"See,  e.g,,  [1],  §10.5  ff 
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of  the  form.  This  matrix  is  symmetric , on  the  strength  of  the  definition,, 
The  latent  roots  of  the  matrix  are  called  the  latent  roots  of  the  quadra- 
tic  form*  In  view  of  the  previous  results,  all  latent  roots  of  a quadra- 
tic form  are  real e 

We  shall  show  that  if  a quadratic  form  is  positive  definite,  its  latent 
roots  are  positive » 

In  demonstration,  let  X be  a real  latent  vector,  belonging  to  X,  a 
latent  root  of  the  matrix  of  the  form*  Then,  since  the  form  is  positive 
definite,  (AX,  X)  > 0o  On  the  other  hand,  (AX,  X)  - X(X,  X) . Thus 

(??j)  X - c 

(x.x) 

But  both  numerator  and  denominator  of  this  fraction  are  positive,  and  con- 
sequently A > 0,  which  is  what  was  required  to  be  proved* 

Let  there  now  be  given  any  real,  non- singular  matrix  A*  Obviously 
B * A5A  is  a symmetric  matrix,  since  B!  - (A?A)?  * A?A"  - A’A  - B» 

We  shall  show  that  a quadratic  form  with  matrix  B is  positive  definite. 
We  have,  ind.eed, 

(?sk)  (BX,  X)  - (A 5 AX,  X)  - (AX,  AX)  > 0 

for  any  real  vector  X„ 

We  shall  establish,  lastly,  that  if  A is  the  matrix  of  a positive 
definite  quadratic  form,  (AX,  X)  > 0 even  for  a complex  vector  X. 

In  proof,  let  X = X + iZ,  where  Y and  Z are  vectors  with  real  compo- 
nents 6 Then 
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(AX,  X)  = (iff  + iAZ- , T + iZ)  » 

(7*1)  * (AY,  Y)  + ±(AZ,  Y)  - i(AY,  Z)  + (AZ,  Z)  = 

- (AY,  Y)  + (AZ , Z)  > 0 

because  (AZ,  Y)  - (Z,  AY)  - (AY,  Z) . 

In  complex  space,  instead  of  the  quadratic  form  one  deals  with 
an  Hermitian  form,  an  expression  of  the  type 


(?»m) 


a,.,  x.x. 
rk  a,  k 


i/t-CL 


under  the  condition  that  a,  . 

Kl 


Ik1 


The  matrix  of  an  Hermitian  form  is  called  Hermitian  (or  Hermitian  symme- 


tric); a linear  transformation  with  an  Hermitian  matrix  relative  to  an  ortho- 
normal  base  is  called  self-conjugate . It  is  obvious  that 


<7  in) 


2 “ (AX,  X) 


To  show  that  all  the  values  of  an  Hermitian  form  are  real,  we  have 
only  to  note  that 


(7fo) 


(AX,  X)  - (X,  A*X)  - (AX,  X)  . 


If  all  the  values  of  an  Hermitian  form  are  positive,  it  is  called 
positive  definite  B 

It  can  be  shown  that  the  latent  roots  of  an  Hermitian  matrix  are 
real , Ihe  latent  roots  of  a positive  definite  Hermitian  form  are  positive , 

^See,  e„ge,  [1],  §30,9 


9 6 The  redaction  of  a matrix  to  diagonal  form.  Let  us  consider  the 

matrix  A all  of  whose  latent  roots,  A-, , •••  s are  distinct,  and  the 

transformation  A connected  with  it  with  respect  to  the  initial  basis  . It 

will,  have  n distinct  latent  vectors  X.  , *»*  , X . wTe  shall  show  that  the 

i5  5 n — — 

vectors  X- , • • a , X are  linearly  independent . 

Assume  the  contrary:  let  the  vectors  X-, , , X^  be  linearly  depen- 

dent. Without  detriment  to  the  generality  we  may  assume  that  the  vectors 
X- , •••  , X,  , 'where  k < n,  are  linearly  Independent,  and  thus  that  the 
vectors  X,  are  linear  combinations  of  them..  In  particular, 

let 


(8) 


K 

X ■ * 2 c.X.  s 

n « - x a.  9 

i-l 


then 


( 8a) 


AX 


n 


k 

A _ 2 cA 
1-1 


k 

2 X .c.X. 

. iii 


On  the  other  hand, 


(8b) 


AX  4 X - 2 c.X. 

m n n n . „ nil 
i-l 


wnence 


(9) 


2 (A  - X.)  c.X.  » 0 . 

. - ' n x x x 
1 .SJL 


But  r K i on  assimption.  Bins,  since  the  vectors  X.  are  linearly  in- 
dependent, all  the  coefficients  c.  equal  zero,  and  therefore  X_  - 0, 
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which.  contradicts  the  definition  of  a latent  vector.  So  the  vectors 
X,,  X-j,  »•*  , X.  are  linearly  independent.  Let  us  adopt  them  as  a new 
basis  of  the  space.  With  respect  to  the  new  basis  the  linear  transforma- 
tion A will  be  connected  with  a matrix  whose  columns  are  composed  of  the 
coordinates  of  the  vectors  AX.,,  AX g , } AXr  with  respect  to  the  basis 


Xl>  X2'* 
But 


, X . 
’ n 


(9a) 


«v  * XA  » 


’ X“I' 


and  the  matrix  of  the  transformation  of  the  new  basis  will  consequently  be 
diagonals  f\>  ' 

So  the  linear  transformation  A has,  with  respect  to  the  initial  basis, 
the  matrix  A,  and  with  respect  to  the  basis  of  the  latent  vectors,  the 


diagonal  matrix  [Ap 
been  noted  above. 


Accordingly,  on  the  strength  of  what  has 


(10) 


v_1w  - [A^  ,Xn]  , 


where  V is  the  matrix  whose  columns  are  the  coordinates  (with  respect  to 
the  initial  basis)  of  the  latent  vectors. 

Observation.  If  the  latent  roots  of  a matrix  are  cf  multiplicity 
greater  than  one,  but  to  each  latent  root  there  correspond  as  many  latent 
vectors  as  it  has  multiplicity , the  matrix  may  also  be  reduced  to  diagonal 
form.  This  will  be  the  case,  for  example,  with  symmetric  matrices?  it 
can  be  proved  that  to  each  latent  root  of  a syrmne trie  matrix  there 
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of  the  latent  root a Moreover,  the  linearly  independent  latent  vectors  be- 
longing to  a single  latent  root  may  be  subjected  to  the  ortho gonalizing 
process,,  We  have  seen,  too,  that  the  latent  vectors  of  a symmetric  matrix; 
that  belong  to  distinct  latent  roots  are  mutually  orthogonal*  Thus  for  a 
symmetric  matrix  it  is  possible  to  construct  an  orthogonal,  system  of  latent 
vectors  forming  a basis  for  the  whole  space* 

The  question  of  the  transformation  of  a symmetric  matrix  to  diagonal 
form  is  closely  connected  with  the  theory  of  quadratic  forms* 

10  e J-kg  latent  roots  and  latent  vectors  of  similar  matrices  * It 
has  been  established  that  similar  matrices  have  identical  characteristic 
polynomials,  and  consequently  identical  spectra  of  latent  roots. 

We  have  explained  the  geometrical  cause  cf  this  circumstance,  vis0s 
similar-  matrices  maybe  regarded  as  matrices  of  one  and  the  same  transforma- 
tion, referred  to  different  bases.  Therefore  the  latent  vectors  of  similar 
matrices  are  columns  of  the  coordinates  of  the  latent  vectors  of  the  trans- 
formation under  consideration,  with  respect  to  different  bases,  and  are 

-1 

thus  connected  by  the  relation  x?  s C x,  C being  the  matrix  of  transform 
mat  ion  of  coordinates*  This  circumstance  may  be  verified  formally!  if 


be  the  given  polynomial 9 Then  the  latent  roots  of  the  matrix  will  be 


Ax  - Ax,  (cf^AC)  (Cf1*)  « A(C  Xx) 


11*  latent  roots  of  a polynomial  in  a matrix*  Let  A be  a matrix 

with  latent  roots  oee  , 


- 6?  ~ 


This  is  readily  established  for  a matrix  all  of  whose  latent  roots  are 
distinct.  Indeed,  such  a matrix  can  be  reduced  to  diagonal  form  by  a simi- 
larity transformation: 


(10. a) 


a ceordingiy , 


(10  .b) 


Eut 


(10  .e)  Cj)(  ' A 


1! 


AnJ)  ’ 


which  follows  from  the  fact  that 


(10  .d) 


onsequently 

, ••• , x ,) » 2 

k"o 


(10 .e) 


k 
n I 


m 

Z 


m 


1 k 

a,  A=,  , * * • 

, K l5  . 

kso  kso 


Tims  the  matrix  (f( A)  is  similar  to  the  matrix  1 --f  ( A , ) s , y ( A n)  j 
and  accordingly  its  latent  roots  are  ^ ( X .1; ) , (j?(A?),  ° - ° , ^f(X  9 


Q «E  ,D . 
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This  result  remains  true  for  any  matrix:,  of  which  one  may  readily  con- 
vince  oneself,  for  example,  by  considerations  of  continuity, 

k k 

We  particularly  note  that  the  latent  roots  of  the  matrix  A are  A . 
12.  The  normaliz ation  of  the  latent  vectors  of  a matrix.  The  second 
group  of  orthogonality  relations.  Let  A be  a r-eal  matrix  whose  latent 


roots,  A , Ari,  A , are  distinct,  and  let  X,  , X~,  , X be  the 

JL  n X d,  TL 

latent  vectors  corresponding  to  than.  As  we  saw,  the  transposed  matrix, 

A*,  has  the  same  latent  roots.  Let  X^  , X*.,  • - • , X(.  be  the  latent  vectors 
of  the  matrix  AT,  and  their  enumeration  be  so  chosen  that  X.  and  X^  belong 
to  complex  conjugate  roots.  We  established  above  that  the  following  or- 
thogonality relation  holds?  (x! , X,)  ® 0 for  i ^ j.  We  shall  show  now 


that,  having  chosen  X,  , X,» 


, X^  in  any  manner  (they  are  determined  but 


for  a numerical  multiplier),  we  may  norm  the  vectors  X. 

(xj!s  x±)  - I. 

In  demonstration  of  this,  the  vectors  X,,  X9, 


l9 


, Xn  so  that 


X are  known  to 
n 


be  linearly  independent,  and  they  accordingly  form  a basis  of  the  space. 


Resolve  X.  in  terns  of  this  basis? 
r 


x: 


Txxi  + 


Xo  + 


+ TI 

0 n 


n n 


j r » 

Forming  the  scalar  product  (X.,  X- ) , we  obtain 

,-L  2. 


(x!.  x!) 


(10  .g) 


h(xi>  V + — + Ti(xi  + - + rj*±. x 


n 


V 


1 
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whence  we  conclude  that  (x! , X.)  - ©<.  / 0 , for  (l!  <,  X. ) > 0B 

XX  XX 

Adopting  instead  of  the  vectors  X,  9 •••  3 XVj  the  vectors 
X.,.  5 we  arrive  at  the  required  normalization^ 


since 


(10  Ja) 


From  the  relations  of  orthogonality  and  normality  set  forth  above,  we 
may  extract  another  group  of  relations  between  the  components  of  the  latent 
vectors  of  the  matrix  A and  its  transpose » 

Form  the  matrices 


(10  .i) 


X5 


All  x2i  *°*  xni\ 

? ? 7 1 

*12  22  Xn2 


V 


*1 


t ? f 

« • e x 

n d.n  nn 


and  X 


Tl  X12  * * * 

X~,  8 * * X, 


\ 


2n 


5 / 


x 


x 


nn 


' \ 

Tne  columns  of  the  matrix  X are  composed  of  the  components  of  the  vectors 
X=j  j 88e  , X,.. . The  rows  of  the  matrix  Xs  are  composed  of  the  numbers  complex- 
conjugate  with  the  components  of  the  vectors  X=j  s 3 X^ . (We  observe 
that  the  numbers  that  are  the  complex  conjugates  of  the  components  of  the 


vectors  xl  3 688  3 X.';  are  the  components  of  the  vectors  xj , 8 8 8 3 xj. , and 

will  also  be  the  latent  vectors  of  the  matrix  A?  that  belong  to  the  latent 

roots  A..  , 88  • , A 6 Thus  the  i-th  row  of  the  matrix  Xs  and  the  i-th 
Is  3 n 


column  of  the  matrix  X are  composed  of  the  components  of  the  latent  vectors 
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of  the  matrix  A5  and  A belonging  to  the  same  latent  root  A,  and  not  to 
latent  roots  that  are  complex  conjugates  of  each  other.) 

It  is  readily  seen  that- 


.) 


X'X  - I 


for  we  have  the  element  of  the  i-th  row  and  j-th  column  of  the  matrix  X!X 
n ^ 

equalling  2 jl  .x,  . 33  (X.,  xl)  33  v where  is  Kronecker 1 s delta; 

k-1  “-^3  3s  3 li5 

, i / j A 

ij  - . I . Thus  X!  and  X are  mutually  inverse  matrices , and  ac- 

cordingly XX 5 is  likewise  equal  to  I.  This  gives  a second  group  of  ortho- 
gonality and  normality  relations  between  the  latent  vectors  of  the  matrices 
A and  A ’ 5 viz , ; 


(lrc 


XilXjl  + X12Xj2  + 


%o  ,1.'  + x.~x.!«  + 

il  il  i2  x2 


+ X.  X. 

in  311 


+ X.  X. 
m in 


1 


Thus  for  a matrix  of  the  second  order  the  ordinary  conditions  of  orthogon- 
ality may  be  written  in  the  form  (we  preserve  only  one  index  of  the  compo- 
nents of  the  latent  vectors,  designating  the  first  components  by  the 
second  by  7) ; 


(12  0a) 


XyX^  + I - 0 

+ y«y^’  - 0 , 

d 1,  C,  JL 
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and  the  normality  conditions 


(12  .b) 


v0  nr*  4* 


The  new  relations  will  be 


(12  .c) 


Observation . In  case  the  latent  roots  of  the  matrix  are  multiple,  and 
to  each  latent  root  there  correspond  as  many  linearly  independent  latent- 
vectors  as  the  multiplicity  of  the  root,  these  indicated  properties  of  the 
latent  vectors  hold  as  before* 


We  have  proved  above  that  if  a matrix  has  n distinct  latent  roots, 
it  may  be  brought-  into  diagonal  form  by  a similarity  transformation*  Given 
the  presence  of  multiple  roots,  however,  such  a transformation  may  not  al- 
ways be  possible*  Nonetheless  the  question  whether  the  form  may  possibly 
be  rendered  more  simple  via  a similarity  transformation  can  well  be  pute 
The  problem  is  equivalent  to  discovering  a basis  with  respect  to  which  the 
linear  transformation  connected  with  the  given  matrix  would  have  a matrix 
of  simplest  form,  and  the  latter  proves  to  be  the  Jordan  canonical  form* 
Proof  of  a fundamental  theorem  to  the  effect  that  any  matrix  may  be 
brought  into  the  Jordan  canonical  form  by  a similarity  transformation  is 


§U.  THE  JORDAN  CANONICAL  FORM 


- 72  - 


rather  complicated  and  we  will  not  dwell  on  it  here.  The  gist  of.  it  is 
stated,  for  instance,  in  [2],  i3.l6|  a scholarly  presentation  is  available 
in,  e„g„,  [3],  Chap.  ?-YX.  We  shall  limit  ourselves  to  a description  of 
this  canonical  form. 

A matrix- -most  commonly  a submatrix““Of  the  following  form  is  called  a 


On  its  principal  diagonal  the  single  number  A,  is  eveiywhere  to  be  found ; 
directly  under  the  diagonal  (in  the  subdiagonal)  are  disposed  elements  that 
are  all  units;  all  the  rest  of  the  elements  are  zero. 

A canonical  box  cannot  be  simplified  by  utilizing  a similarity  trans- 
formation. It  is  obvious  that  a canonical  box  has  the  sole  multiple  latent 
root  X. . It  may  be  easily  verified  that  a canonical  box  has  only  one 

latent  vector.  The  minimum  polynomial  of  a box  coincides  with  its  charac- 

m. 

teristic  polynomial , viz,,  it  equals  (\  - A,.)  " where  is  the  order  of 
the  box.  The  J or  dan  (classical)  canonical  form  is  a quasi-diagonal,  matrix 
composed  of  canonical  boxes s 
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\ 0 0 •••  0 

i X1  o • o 

0 1 A,  • » • 0 


(2) 


X o o * • • o 

s 

i A o o 

s 

o i A •••  o 

s 

tt  O 6 d • • o 

o o o • • * X 

s 


It  is  admissible  that  the  same  number  A^  appear  in  several  canonical  boxes. 

All  the  numbers  appearing  in  the  different  boxes  are  latent  roots 

of  the  canonical  matrix , and  the  multiplicity  of  the  latent  roots  equals 

the  sum  of  the  orders  of  the  boxes  in  which  it  figures  as  diagonal  element. 

In  proof  of  -this,  by  the  theorem  concerning  the  determinant  of  a qua  si -di  agon  a 

matrix,  the  characteristic  polynomial  of  the  canonical  matrix  equals  the 

product  of  the  characteristic  polynomials  of  the  separate  boxes,  each  of 

m. 

them  equal  to  (A  - A . ) x where  A,  is  the  latent  root  and  m.  is  the 
order  of  the  i-th  box.  Hence  follows  directly  the  statement  under  proof. 

The  determination  of  the  Jordan  boxes  for  a given  matrix  A presents 

m. 

certain  difficulties . The  characteristic  polynomial  "f~[  (A  - A ) L 
coincides  with  the  characteristic  polynomial  of  the  original  matrix,  and 


“ Ih  - 


it  is  consequently  possible  to  find  it  without  knowing  the  canonical  matrix 

itself 5 Nonetheless,  a knowledge  of  the  characteristic  polynomial  still 

does  not  make  possible  the  complete  determination  of  the  canonical  form,  for 

to  a latent  root  A,  of  multiplicity  k,  there  may  correspond  several.  Jordan 

i i 

boxes  containing  this  number  as  a diagonal  element,  and  regarding  them  only 
the  sum  of  their  orders  will  be  known,  not  the  order  of  each  box  in  particu- 
lar e If  the  canonical  form  is  to  be  fully  determined,  a knowledge  of  the 
’■elementary  divisors’’  of  the  matrix  must  be  drawn  upon. 

Designate  by  D^( A ) the  greatest  common  divisor  of  all  the  minors  of 
the  i-th  order  of  the  determinant  |A  - Al|.  In  particular,  D ( X)  co- 
incides with  the  characteristic .polynomial o It  can  be  proved  that  all 
D±(A),  as  D ( A),  are  general  for  the  class  of  similar  matrices.  It  can 
be  proved,  moreover,  that  D.  . ( A ) divides  D.  ( A ) A 
Put 


(3) 


D.  ( A) 


D.  .(A 


obviously 

(1) 


n 


V*>  ■ TT  *,(*> 


i-1 


It  turns  out,  moreover,  that  E ( A ) 


D ( A ) 


n 


Ti 

is  the  minimum  poly- 

U_  i \ A / 
n=l 


nomial  of  the  matrix. 

Resolve  E . ( A ) into  linear  factors.  Then 


"See  [3],  p.  23  ff»,  [5]  50  ff 
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m . 


(5) 


e.(A)  - FT  (X,  - >)  13 
1 3=1  J 


n 

Here  s denotes  the  number  of  distinct  latent  roots,  2 m.  . ® k.J 
s n i"X  J 

2 2 m.  . - n0  It  is  obvious  that  among  the  exponents  m.  . only  some  will 

j=l  i=l 

different  from  zero  „ 


Translator’s  Note . It  may  be  helpful  to  the  student  to  have  in 
extenso  a synopsis  of  the  relations  of  these  important  entities s 


D ( A ) 

■ 07  ' En(  A )s 


D . 
n-1 

Thus 


|D»  is  the  (i“th)  determinantal  divisor  of  D^j 
|E,=  is  the  (i-th)  invariant  factor  of  0 


Dn(A) 


Cn-1(A) 


En(A) 


and,  in  sequence. 


D ) “ D „(  X)  0 E .( \)  , 

n~l  / rt"2  n--lN  ‘ 5 

*©©e©o«»®do 

d3(a)  - d2(70  . e3(A)  s 
d2(  A ) - D1(A)  - e2(A)  , 
dx(A  } - - 


Reversing  our  view  of  the  developments 


M > ) 
d2U) 
d3(X) 


, 

Di(  X)*E2(  A) 
D2(  A)«e3(A) 
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= iij(  A )*Eo(  A) 


A)*e2(  X)-e^(A)  - flE.(X) 


i-1 


n 


(6)  r>n(  A ) s d - ( A ) *sn(  A ) » Ej(A) e ( A ) = FT  A ) * 

i=l 

Consider  now  that  each  E.  is  factorable  into  linear  factors 

1 

holding  the  s distinct  latent  roots  These  factors  may  be  mul- 

tiple, or  may  not  appear  in  the  given  E.,  (multiplicity  = 0): 


m.,  m.0 

e.(A)  - (A  - A1)  xl-  (A  - A2)  I: 


m. 


( X - X ) 15 

s 


(7) 


m. 


(A  - A .)  13 

3=1  3 


He i e m.  . is  the  power  (perhaps  zero)  of  the  j-th  distinct  linear 
factor  of  E^;  there  are  s of  such  powered  prime  factors,  each  of 
which  latter  is  called  an  elementary  divisor  of  D ( X ),  since,  using 


(7)  in  (6),  we  have 


n 


ns  m.  . 

d <X)  - nyx)  - n n (x  - K)  13 

i«X  1 i-1  j-1  3 


Since  there  are  n latent  roots  of  D (A)a  some  perhaps  multiple, 
n s 

2 2 m.  . = n. 

i-1  j*l  13 

Since  the  linear  factor  for  the  j-th  latent  root,  ( X - X.)9 

J 

may  appear  in  more  than  one  E.,  —a  total  of  k.  times,  in  fact,  we 
n 

have  2 m.  . = k . e 
i-1  13  0 
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It  is  usual  , on  behalf  of  greater  generality , not  to  limit 

one’s  concern  to  D (A)  - JA  - A I i } but  to  conceive  D^(  A ) as  a 

matrix  all  of  whose  elements  are  polynomials— -of  whatsoever  degree— 

with  coefficients  in  some  specified  field,  rather  than,  as  here,  a 

matrix  whose  diagonal  elements  only  are  polynomials,  and  those  of 

the  first  degree,  'Hie  above  treatment,  otherwise  unaltered,  then 

has  as  its  subject  such  a generalized  D,  a(*),  a.  ’’lambda  matrix” , 

as  it  has  come  to  be  spoken  of. 

m.  . 

The  binomials  ( A - ^ are  known  as  the  elementary  divisors  of 

jA  - A I),  and,  by  extension,  as  those  of  the  matrix  A,  A knowledge  of  the 

elementary  divisors  permits  us  to  construct  the  canonical  form,  viz.;  the 

Jordan  boxes  are  constructed  by  starting  from  the  number  A ,,  and  the  orders 

3 

of  these  boxes  are  equal  to  the  exponents  rm  . The  number  of  boxes  contain” 

irig  A . equals  the  number  of  exponents  m,  ..  not  equal  to  zero , 

3 <d 

In  case  the  elementary  divisors  are  linear,  i.e,,  if  all  the  non-zero 
exponents  m. . are  equal  to  one,  the  Jordan  boxes  degenerate  into  diagonal 
elements,  and  the  canonical  form  turns  out  to  be  simply  a diagonal  form, 
wherein,  of  course,  a single  latent  root  will  appear  as  of  ten  as  a diagon- 
al element  as  it  has  multiplicity  as  a root  of  the  characteristic  equa- 
tion. 

The  converse  is  also  obvious,  for  it  is  clear  that  if  a matrix  can 
be  brought  to  diagonal  form,  its  elementary  divisors  are  linear.  There- 
fore matrices  with  distinct  latent  roots,  as  also  symmetric  matrices, 
have  linear  elementary  divisors. 
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m.  , 

If  all  the  elementary  divisors  (X  - 4^)  ^ are  relatively  prime 
(which  occurs  only  in  case  _(A  ) = l) , each  latent  root  appears  in  only 
one  canonical  box?  and  the  order  of  the  box  equals  the  multiplicity  of  the 
corresponding  latent  root.  Only  in  this  case  does  the  minimum  polynomial, 
coincide  with  the  characteristic  polynomial. 

Let  us  now  consider  the  matrix  transforming  the  given  matrix  into 
canonical  form.  With  this  object  in  view  we  introduce  into  the  discussion 
a linear  transformation  connected  with  the  matrix  A with,  respect  to  the 
initial  basis.  Then  the  columns  of  the  transforming  matrix  will  be  compo- 
nents of  the  vectors  of  that  basis  in  which  the  linear  transformation  in 
question  is  described  as  a canonical  matrix. 

Let  this  canonical  matrix  have  the  form 


nO 

i -s 


fj)  ('■  ) 

and  the  order  j£  A ec-.  1:  re.  ; let  1C  V.. " ' , 


t’:  ' ' he  tl-e 
m 


h\ 

* » * rr  ■' 

$ *v;  j> 

.vf.Tj 


7 0 * .V-' 1 


..  on  t i - -■  j-  onfirij.  as  tor  the 


Jr. 


• • ' # EE  .A.  * : V,'L 

/ *■  '-'•“T  ‘ „ - •n 


(10) 


/ ...n 


irO)  . A «0)  + i(0) 


~jL 


,(iO 


o f (T 

i-l 

\ n(r) 


1:;"'  * A lr. 


¥e  se  bl  tors  c =1  - s r e 3 a i;  ent 

rectors  matrix,  perl  , b<  proved  thal  ra.th  this 

all  111 naep  nc  3 of  the  matri  are  exhausted,  and 

consequently  bh<  number  )i  rl;  iependent  j it  vectors  of  the  givei 

atrix  e . • _ . the  numb  . L > i Its  canond  cal  fom.  In 

particular,  th  ixo 3 ' - latent  vectors  belonging  to 

the  givei  late  it  roe  tl  uni  ra  boxes  cc  taining  this 

>ot.  II  i not  of  . V ' ■ n the  latent  root,,  and  is 

e [aal  bo  ' ix  . V i 3,  a 3 in  < , all  - . contain- 
ing the  given  3.atent  i rder  1,  i9.e.,  when  .t  e c r i spc  nc  ing 

ele- lent,  tiviro  x are  linear. 


_ F 


§5>.  THE  CONCEPT  OF  LIMIT  FOB  VECTORS  AND  MATRICES 


(Ji) 


Let  a sequence  of  vectors  X 

Xl)s  , (k) 

«♦  u ' r>  o e <? 

'll 


(1)  ,-(2) 


M 


_u 


with  c o mpo  nents 

, :t.v;  ‘J)  3 "»■*  be  given.  If  a limit 


Xk) 


(k) 


exists  for  each  <y.  moonents  list  xf  ' « ;c,  . the  vector:  X with  components 

« -h  * 

iC-400 

-v-w  y ' — 3 


A 


....  Y 

) •)  -x 


\K) 


and  the  £ ice  i I s sai  - bor  X.  This 

written  in  t .e  f-rn:  Xv  " — > X e,  1:L.  XK  ' - X. 


►<p 


in 


; s.n.i.:  ' ashi-  , gi  e i a _ quence  of  square  .matrices 

Aj\  A ■ K •••  , A , •••  with  I (ai'P),  **”  , (a-^), 

— <. 

( vs 

he  m • i ...  . -•  i . . if  - bhe  ] of  the 

■h  k— >oo  ^ 

sequence,  if  all  these  l±r,tLh  eislst. 

In  n fitb  ? definition  of  a limit,  an  infinite  series 


« • * is  said  '.  o b • convergent  if 


( i ) (?) 

vrrsors  X • + X'"  + • • - 

, h ) f 2) 

lim  (X'-'  + X ■ + J 
k — v.n 

give!  series.  s ::-r,  and  sufficient  for  the  convergenc 

of  a seri  rector  th  .1]  s 2 composed  of  their  corresponding 

components,  i „e * components  bearing  the  same  indices,  converge,  the  sms 
of  the;  . ies  are  the  components  of  the  sum  of  the  series  of  vectors . 

The  : . of  the  convergence  of  a series  of  matrices  is  defined 
analogously , 

In  appli  I Lest  . ...  11]  i it  ts  it  bo  judge  not  only  the 

convergence  of  a sequence  or  series,  but  also  to  judge  the  rapidity  of 
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this  convergence  • With  this  object  in  vietr,  th:  in:  .eduction  of  the  norms 
of  vector  ar  aatiiees  is  quite  . . „ A norm  may  be  introduced  in 


diff 

arent  navs  „ 

and 

in  difi 

most 

con 

. 

Go  neral 

•iy, 

the 

norm  of 

!■  rr  11 
1.  A II 

satim;' 

ying 

Lxi- 

3 fc-llo’c -r; 

1) 

! ]7i 

. i a ( 

! > 

0 for  X 

2)  ! 

i \ 

i * 

P.  f j h'  j 
! • 1 i r i 

3) 

i ]x 

i ! 4 1 

i j 


From  requirements  2) 


(1) 


indeed,  r<  ' i 


...  equality41) 

3)  it  is  readily  deduced  that 


i I- 


. : i -s  Si! 


•>  Sir ! ! 


! lx!  * 

M il 


(2) 


: j \r  f i 

1 PM  I 


i Hr 

X 


”1,  45  1 1 - - I 


imi 


md  '’here fore 


(a) 


j } ' _ Y ! : 2k  ; ) 'r  I j i |v  j j 

I ! 1 " I i1  i 1 ! I i i 


(U) 


i !-  - I I 


1 iy  M „ ; l-r  i l 
i I i 1 I 1 I ® 
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Consequently 

(5) 


||X  - Y|  I » ! |x|  I - I |Y|  I | „ 

We  shall  henceforth  make  use  of  the  following  three  ways  of  assigning 


a norms 

if  X s 

f " 

1 |x| 

ill" 

(6) 

< IIo 

I|I| 

III  ** 

( III, 

IW 

1 III  ! 

max  i. 
i 1 


XHII  " 1*1.1  + 1^2 1 + + lxni 


/ K 1 2*  M2™  !xn  i 


It  is  obvious  that  for  all  three  norms  all  the  requirements  1)  = 3) 
are  fulfilled , 

The  concept  of  the  norm  of  a vector  generalizes  the  concept  of  the 
length  of  a vector 5 since  for  length  all  the  requirements  1)  - 3)  are  ful- 
filled,, The  third  norm  introduced  by  us  is  indeed,  none  other  than  the 
length  of  the  vector „ 

Furthermore 9 it  is  easily  established  that  a necessary  and  sufficient 

(k) 

condition  that  the  sequence  of  vectors  X converge  to  the  vector  X is 
that  | |x^;  — X | 1 0 for  each  of  the  three  norms  indicated.  For  the 

first  norm  this  is  obvious.  For  the  second  and  third  norm  this  follows 
from  the  obvious  inequalities 


llxlij  £ ||x||n  i n ||X|| 


(7) 


IWIj  £ I |x|  |m  « VS|  1x1  Ix 


. 


. 


■ 


' 


_ 


. 


■ 


* 

.. 
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It  is  easily  shown  that  for  convergence  of  a sequence  of  vectors 
X^  to  a vector  X it  is  necessary  and  sufficient  that  J jX  - X^^  | | 0, 

whatever  nom  satisfying  conditions  l)  - 3)  we  may  choose.  Here,  if 
X(k)  X,  ||l(k)M— >||I||,  for  j||X||  - ||Xl-:)||j4  ||x  - X(':)||  --=>  0. 

In  an  analogous  fashion,  the  norm  ox  a square  matrix  A is  a non-nega- 
tive number  A satisfying  -the  conditions 


(8) 


1)  | |A  | ] > 0 if  A ^ 0 and  jjOjj  = 0 ; 

2)  UcAjl  = |e||]A||  5 


3)  | |A  + B||  * | [A||  + 1 1 B J | 


W 


inn  iibii  • 


Just  as  in  the  case  of  the  norms  of  vectors,  the  condition  | |A  "k  ~ A||  - * 0 

/•»  \ 

is  necessary  and  sufficient  in  order  that  A K ' — ■£  A,  and.  just  as  in  the 
case  of  the  norms  of  vectors,  it  follows  from  A^  ’ ,;  A that  | |A^k;  j | — ^ | |A  j 
The  nom  of  a matrix  may  be  introduced  in  an  infinite  variety  of  ways. 
Because  in  the  majority  of  problems  connected  with  estimates  both  matrices 
and  vectors  appear  simultaneously  in  the  reasoning,  it  is  expeditious  to 
introduce  the  nom  of  a matrix  in  such  a way  that  it  will  be  rationally 
connected  with  the  vector  norms  employed  in  the  argument  in  hand.  We  shall 
say  that  the  nom  of  a matrix  is  compatible  with  a given  norm  of  vectors  if 
for  any  matrix  A and  any  vector  X the  following  inequality  is  satisfied i 


(9; 


!-« ! I 4 I Id  I 11*11 


eu 


We  will  now  indicate  a device  making  it  possible  to  construct  the 
matrix  norm,  so  as  to  render  it  compatible  with  a given  vector  norm,  to  wits 
we  shall  adopt  for  the  norm,  of  the  matrix  A the  maximum  of  the  norms  of  the 
vectors  AX  on  the  assumption  that  the  vector  X runs  over  the  set  of  all 
vectors  whose  norm  equals  unity: 


(10) 


max 

I Id  K 


! !•«  1 1 


I -onsequence  of  the  continuity  of  a norm,  for  each  matrix  A this  maxi- 


mum is  attainable,  I0e„,  a vector  X can  be  found  such  that  | jX^  j J - 1 and 

IIAXJI  - ||A||. 

We  shall  prove  that  a norm  constructed  in  such  a manner  satisfies  re-* 
quirements  1.)  - U),  set  previously,  and  the  compatibility  condition „ 

Let  us  begin  with  the  verification  of  the  first,  requirement » 

Let  A f 0e  Then  a vector  X,  j jX  | f - 1,  can  be  found  such  that  AX  f 0, 
and  accordingly  jjAXjj  / 0,  Therefore  jjAjj  55  max  jjAXj)  / 0„  If, 

I |x| l-i 

however,  A - 0,  jjAjj  = max  | jOX  I j ~ 0. 

l|i  IK 

Second  requirement.  On  the  strength  of  the  definition,  | |cA  j j - 


max  j j cAX j 
max  | c j 

llxll-x 


j a Obviously  | j cA.X  j ] = jc  j 
Max  j j - |c|mx|  |AX|  j - jc 


j j AX  j j and  thus 

i |a  ! ! « 


IKII  - 


Let  us  verify,  furthermore,  the  compatibility  condition. 


Let  T ^ 0 be  ary  vectdrj  then  X s 
that  j jx||  - I*  Consequently  |jAY||  » 


— h—  T will  satisfy  the  condition 

||l|| 

||A(||T||x)|!  - ||T||  llAxjj  « 


4 
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Third  requirement,,  7 or  Jie  matrix  A + B find  a vector  X such  that 
I |A  + B 1 1 - 1 1 (A  * B)Xc  1 1 and  Ijl.jj  - 1.  Then  ) |A  + B H - ||(A  + B)Xjj  = 

”!l*?0  * Bre|)  * Uax0|!  ♦ ||m  ||  * 1 J a 1 1 ||xjj  * | |b 1 1 ||x0||  - 

- INI  + 1 |B  I i . 

Lastly,  the  fourth  re  quin  emeu  » Fox  +te  matrix  AB  find  a vector  X 
l|x0l|  ” 1 eai  ||ABXo||  »'  ||AB||.  Then  ||  ,||  - 

* I |a(bxo)  1 1 * I |a  1 1 l|BXo||  i 1 1 A 1 1 1|B|  | Hull  - l|A||  ||b||. 

We  nave  verified  the  is  fact  ion  of  all  four  requirements  and  the 
compatibility  condition*  . matrix  norm  c-e as true ted  in  this  manner  we  shall 
speak  of  as  subordinate  to  the  given  norm  of  vectors  * It  is  obvious  that 
for  any  matrix  norm,  subordinate  .0  wn at soever  vector  norm,  j jl j | s 16 
Let  us  now  construct  matrix  noimr  subordinate  to  the  three  norms  of 
vectors  introduced  above, 

(n)  I.  I |X||  * -max  |x±|  8 

The  matrix  norm  subordinate  to  -ails  vector  norm  : 


rs 


(13) 
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Consequently 


n 


(xu) 


max  llAljS  - max  2 la.. 
|X  | |S1  1 k-1  ^ 


n 


He  shall  now  prove  th^t  max  jjAXjj  is  in  fact  equal,  to  max  2 j 

I |x|  1-1  i k-1 

this  we  shall  construct, a vector  X such  that  | iX. ] f - 1 and  | jAX& 
n n 

- max  2 ja„.,.|e  Letting  2 |a..  | attain  its  greatest  value  for 
i k-1  ^ k-1  iK  « a 

( P 4-  V*  ■rpAi.+  rtw  V f.  %'  ( o)  „ ' a jk  * 


then  taking  as  the  component  xl  of  the  vector  X.  s x. 


a 


3k 


a.v  f 0,  and  xK'}  1 if  t ....  ~ ’ we  have , obviously,  j jx  { | = 1 


jk 

more , 


OS) 


n 

kf1*iK*k 


(o) 


n 

* 2 
k-1 


a. 


'ik 


n 

2 |a 
k-1 


jk 


for  i ^ j 


and 


(16) 


n / s, 

2 a x)w 
k-1  3klC 


n 

2 

k-1 


ajk 


Consequently 


(17 


1 « (o) j 

max  2 a„.,r  " 


n 


k-1 


iC 


2 |a^J  - max  2 j; 


k-1 


i ks 


ik 1 


Thus  Max  j]  - max  2 j a.-  L Q.E.D, 
° i k-1 


(18)  IX, 


1*11 


II 


n 

2 lx. 
i-1  ’ 1 


aik|0  For 

II  3 

i s j,  and 
if 

Further- 


3? 


The  matrix  norm  subordinate  to  this  vector  norm  is 

n 


(19) 


| | A | j - max  2 \ &±k\ 

k i-1 


In  proof  thereof  s let  jjljj 

- ls  then 

n 

n 

n n 

liAXll  - Z i 

i“l 

2 a.,  x,  ] 

, -s  ik  k 1 • 
&--JL 

4 2 2 |a,  I |xj  * 

i-1  k-1 

n 1 

n \ 

n \ n 

(20)  ^ 2 |x.  | 

k-1  ■ * v 

Z Kvl 

iO  / 

n 


max 
k 1- 


s hik! 


n 


Now  let  us  take  a vector  X of  the  following  form:  let  2 |a«,  j 

0 (o) 


its  greatest  value  for  the  column  numbered,  j.  Put  x,;"  3 0 for  k r j 

(o) 

xV"  “ le  Obviously  a vector  constructed  in  this  manner  has  its  norm 
unity  „ Furthermore 


(21)  I |AX  ||  - 2 I Z a. 

1-1  k-1 


(o) 


n 

2' 

ial 


n 


3-3 


-max  2 |aik| 


k k<L 


Thus 


(22) 


n 


max  j j.AJi  | j - max  2 j a,. , 
k i“X 


nun 


(23)  in 


in 


n 

= 2 


K 


(x.'x) 


attain 

and 

equal  to 
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The  matrix  norm  subordinate  to  this  vector  norm  is 


(21|) 


where  X^  is  the  largest  latent  root  of  the  matrix  A'A. 
In  proof,  we  have 


(250 


max  1 1.«  1 1 

I lx  I l-i 


but 

(26)  ] |AX||  ..  . - (AX  AX)  - (X,  A»AX)  . 

The  matrix  A’ A is  symmetric.  Let  be  its  latent  roots 

and  Xp  X^,  •••  , be  the  ortho  normal  system  of  latent  vectors  belonging 
to  these  latent  roots . 

Now  take  any  vector  X with  its  norm  equal  to  unity  and  resolve  it  in 
terms  of  the  latent  vectors? 


X 


C1X1 


c . X =*. 


c X 


n 


Then 

(28) 


Moreover, 


(X,  X) 


2 

+ + 
i 


o 

+ f* = “ 1 

'"n 
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j|AX|  I4*  - (X.  A* AX) 


(29) 


X,  + .<..4 
11  n n 


, c.,  A .X,  + 

» J . X 


+ a A X ) = 
n n n 


A /»""  4-  • * • + A.  c."~  ^ * ! r " ■!» 

ri  r-S  1^1 


* % ■ Ai  • 


For  the  vector  X ■ X«  s 

(30)  ilA^H  ” - (Xx,  k'AXj)  - (X.:  , ^ . 


(31) 

n TT  ^ 

««*;  •-I-J  or1  d 


Bffii|[K  ||  - /AT  , 

|]i|l-i 


We  shall  now  prove  several  theorems  connected  with  the  concept  cf 
limit . 

THEOREM  10  In  order  that  It.  necessary  and  sufficient 

11  the  latent  lulus  less  than  unity. 


Frccf  . Assume  for  sirrplicity  that  the  matrix  A can  be  brought  into 


diagonal  form; 


A C 


-1 


where  a 


X 


7 5 
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A 


A i and 
nj 

m 


n 


are  the  latent  roots  .of  matrix  A.  Then  A~  “ C A 


, m -X 


It  is  obvious  that  - -A"h?  A“,  , '■  1. \ . In  order  that  A"1,  — * 0, 


r* 

a m 


AX',  e * * , A !;i  « In  order  that  A 
n| 


m 


t is  necessary  and  sufficient  that  A'1  — 0,  for  the  which  it  is  in  turn 

have 


ncecssary  and  sufficient  that  the  latent  roots  X_ . 

1' 


^ 5* 


n 


modulus  less  than  unity. 
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In  case  the  matrix  A cannot  be  brought  into  diagonal,  form,  the  theorem 
is  proved  either  with  the  aid  of  considerations  of  continuity  or  by  passing 
to  the  Jordan  canonical  form.  We  shall  not  dwell  on  the  details  of  this 

proof . 

The  conditions  given  In  Theorem  1 are  inconvenient  for  checks  , inasmuch 

as  they  require  foreknowledge  of  the  latent  roots  of  the  matrix  A.  We  shall 

therefore  establish  some  simpler  sufficient  conditions  rendering 

lira  Ara  - 0a 
m->oo 

THEOREM  2.  In  order  that  A3'  — ->  0,  it  is  sufficient  that  ary  one  of 
the  norms  of  A be  less  than  or lity . 

Proof e On  the  strength  of  the  fourth  requirement  of  a norm,  we  have 

(32)  i jAm|  | ^ ||Am-1||  ||A||<  \\tT2\\  i|A|l24...4  ||A|f  . 

Therefore  j |A  j j » 0 if  j ]A  j j < 1,  and  thus  , in  view  of  the  foregoing, 

A “ ^ 0 , 

Combining  Theorems  1 and  2,  we  arrive  at  the  following  results 
THEOREM  3 . The  modulus  of  no  latent  root  of  a matrix  exceeds  any  of 
its  norms . 

Proof.  Let  j |A  j j = a„  Consider  a matrix  B - A,  where  € is  ary- 
positive  number.  We  have 


(33) 


B 


a+c  “ ^ 


m 


amd  accordingly  B “ > 


On  the  strength  of  Theorem  1 its 
Bur  it  is  obvious  that  the 


latent  roots  have  modulus  less  than  arlty. 


< 

i 


' 
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latent  roots  of  the  matrix  B equal  \ 3 where  A^  are  the  latent  roots 
of  the  matrix  A.  Thus  < 1,  i.e,,  j A j < a * €e  Since  € may  be 

taken  arbitrarily  small 3 j A 4 } ~ a „ 

THEOREM  lie  in  order  that  the  series 

(3  k)  I + A + • • • + A'  “ + *’• 

converge,  it  is  necessary  and,  sufficient  that  An'  0 a£  m co  . In  such 
a case  the  sum  of  series  (3k)  equals  (I  - A) 

Proof  e The  necessity  of  this  condition  is  obvious . We  shall  show  that 
it  is  sufficient. 

On  the  s trength  of  Theorem  1,  all  latent  roots  of  the  matrix  A are 
less  than  of  unit  modulus. 

Accordingly 


vi5) 


|I  - A I / 0 


and  therefore  (I  - A)  L exists. 
Consider  the  identity 


(36)  (I  + A + A ' + •••  + A )(1  - A)  - I - Ak+1  . 

s “1 

P os  tmul  tip  lying  it  by  (I  - A)  , we  obtain 

(37)  I + A + k + + Ak  - (I  - A)~]  - Ak+J"(l  - A)"1 


whence  it  follows  that,  as  k — ce 


i 

* 


«i 


I + a + . . . + Ak  — = » (l  - A)  1 


Ak+1  . 

sxnce  A 3 , 


Thun 


(39) 


+ A + 


A + 


* (I  - A)”" 


which  is  what  was  required  to  be  proved* 

In  the  light  of  Theorem  1,  the  necessary  and  sufficient  condition  for 
the  convergence  of  the  series  '3  U)  is  the  inequality  | X . j < 1 for  all  latent 
roots  of  the  matrix  A * A sufficient,  token  of  convergence,  in  view  of 
Theorem  2 , is  the  ineqi  | JaJ|  < 1,  whatever  one  of  the  norms  be  em- 

ployed. Given  that  this  condition  is  satisfied,  it  is  easy  to  give  the 
following  estimate  of  the  rapidity  of  convergence  of  the  series  (3Jjl)  s 
THEOREM  5.  If  | |A| | < 1, 


UjOJ 


11(1“  A)  1 - (I  + A + 


ak) 


|A|| 


k+1 


Proof.  We  haves 


(U) 


(i  - a)-1  - (i  + a + 


.k+1  . .k+2 

+ A ) “ A + A + • *• 


whence 


(n2) 


%~1 


(I  - A;  " - (I  + A + 0 ■ • + A ) | j ^ 


* l!-Ulk+1  + llAllk+2  + • 


|A 


i k+1 


i-l  !a 


and  the  theorem  is  proved 
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in  finding  a proof  for  the  general  case. 

We  consider  ourselves  obligated  to  note  in  conclusion  that  in 
the  method  here  proposed  we  have  utilized  the  ideas  of  a method  of 
successive  approximations  for  the  solution  of  systems  of  linear 
equations  expounded  in  an  unpublished  work  of  A„  M.  Lopshits. 
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